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Random processes are of great and ever increasing interest to 
scientists and engineers working in various branches of radio 
physics. This is quite understandable, since at the present stage 
of technological development, random noise constitutes the chief 
obstacle to further improvement of certain engineering devices. 
In fact, a current problem in many fields of radio physics is that of 
evolving techniques for analysis and design of equipment which 
is immune to noise. Convincing proof of the importance and 
urgency of this subject is afforded by the large number of books 
devoted to it, published both in the U.S.S.R. and elsewhere, 

The present book, in two volumes, is concerned with a variety 
of topics which have been left to one side, as it were, but which are 
nevertheless of great practical importance. In particular, we have 
in mind problems involving the effects of noise on electronic 
relays and vacuum-tube oscillators. Recently, there has been 
increased interest in the operation of switching devices and self- 
excited systems, not only because of the development of methods 
of coherent radar detection, but also in connection with such 
problems as increasing the accuracy of time measurements, 
designing particle accelerators and computing machines, etc. 
However, the journal literature in these new fields of statistical 
radio engineering is hard to penetrate. In the present book, noise 
in relays and oscillators is studied in an organized fashion, from a 
rather general point of view. Material that can be found in the 
literature has been systematized, and the results of some original 
investigations by the author have been included. The physical 
treatment of the results obtained may be a bit sketchy, but it 
‘seems to us that brevity coupled with wide coverage is justified 
jn this instance, since the rapid growth in the total number of 
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published papers has increased the need for some kind of survey, 
even one resembling a handbook. 

From a mathematical point of view, the topics studied here 
involve nonlinear transformations of random functions, mostly 
transformations with memory. Certain mathematical methods are 
worked out in a form suitable for application to problems of radio 
engineering, and are then exploited systematically. As the reader 
will discover, the following methods turn out to be effective: 

1. Reduction of a given process to a process without aftereffect, 
followed by application of Markov process techniques, in parti- 
cular, the Fokker-Planck equation; 

Linearization of the original equations, which allows us to apply 
correlation theory in the linear approximation; 


Won =e ND 


The quasi-static approach, which allows us to reduce a non- 
linear transformation with memory to one without memory. 


| The treatments in the available literature usually deal only with 
the correlation theory, or only with the theory of Markov processes. 
However, to solve concrete problems, one must be able to go from 
the methods of either theory to those of the other, choosing the 
best methods for solving the problem at hand. For this reason, 
we pay special attention to the problem of when, and in what 
sense, the theory of Markov processes can be applied to actual 
fluctuations, which are originally described in terms of the correla- 
tion theory. 

Part | of Volume I is devoted to a review of the mathematical 
results used later in the book. In many places, it has seemed 
appropriate, for the sake of brevity, to depart from the usual 
presentation of the basic concepts and results of probability 
theory. The reader who is not familiar with this necessary back- 
ground material can refer to any one of a number of textbooks, 
Those already acquainted with elementary probability theory will 
find that Part | contains a different treatment of certain well-known 
topics, in addition to some new results. In fact, Part 1 ought not to 
be devoid of interest even to mathematicians, provided, of course, 
that they make due allowances for the heuristic level of rigor adopted. 
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In Volume I, Part 2 and in Volume II, we systematically use the 
methods just enumerated to solve specific problems of statistical 
radio engineering. At the same time, we try to explain the conditions 
under which the various methods of analyzing noise phenomena 
are applicable, Perhaps some of the methods are given in more 
detail than is necessary for immediate applications, but this is done 
with a view to further development of the tools needed for future 
work, It is reasonable to expect that the progressively more compli- 
cated problems which come up in engineering practice will require 
the use of progressively more sophisticated methods of analysis. 
The author will regard his task as accomplished if this book 
serves in some measure to broaden the theoretical outlook of 
radio scientists, by helping them master certain methods which 
they may find novel at first. Although the material given here is 
intended primarily for specialists in the field of statistical radio 
engineering, it should also interest scientists working in other 
fields where similar statistical methods can be profitably used. 

As a rule, no attempt is made to cite the papers where various 
results appear for the first time. This is because any attempt to 
make systematic source references would be likely to entail errors, 
unless one wishes to become involved in special bibliographic 
research. 

Finally, the author would like to take this opportunity to thank 
V. I, Tikhonov, who suggested that the present book be written 
and rendered substantial assistance, Y. L. Klimontovich, who in 
his capacity as editor made many valuable suggestions while the 
manuscript was being prepared for the printer, and I. G. Akopyan, 
who constructed the figures appearing in Volume II, Chapter 9, 
The author would also like to thank (among others) S. P. Strelkov, 
S. D. Gvozdover, B. R. Levin, S, A. Akhmanov, Y. M. Romanovski 
and P, S. Lande, for their help with the manuscript. He also 
expresses his gratitude to Dr. R. A. Silverman for undertaking 
to translate and edit the English-language edition of this book, 
thereby greatly enlarging its prospective audience. 


R. LS. 


Translator’s Preface to Volume | 


_ The present book is the first of two volumes dealing with 
various topics in the theory of random noise. This material is for 
the most part not available elsewhere in the English-language 
literature, and is in fact mainly due to Dr, Stratonovich and his 
co-workers. Speaking as one who has long been interested in 
noise theory, I feel confident that a large audience of scientists and 
engineers will find the contents of this volume interesting, timely, 
and most important of all, highly original. 
_ I have worked through all the mathematical derivations in the 
book, and in the process have detected and corrected some typo- 
graphical errors which had survived in the Russian edition. Various 
tmall improvements that occurred to me as I studied the book have 
been incorporated without special comment. In reply to my 
queries, Dr. Stratonovich has cleared up a few stubborn points 
that I was unable to resolve by myself, and the present edition 
‘benefits from these clarifications. For example, the supplement to 
‘Chapter 4, giving the details of a rather tricky argument, stems 
rom a letter by the author. 
j Passages appearing between boldface asterisks were printed in 
ibrevier originally, They relate to certain side issues which can be 
‘omitted without loss of continuity. Finally, bearing in mind the 
‘needs of its readers, I have suitably modified the list of books cited 
‘in the Bibliography. 
, R. A. S. 
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PART 1 


General Theory of 
Random Processes 


CHAPTER 1 


Random Functions and Their 
Statistical Characteristics 


41. Random Variables 


Roughly speaking, a random variable € is characterized by the 
fact that instead of knowing its precise value, we only know how to 
obtain various values of € under certain experimental conditions 
which are fixed, stable and physically well-defined. As a result of 
unknown factors, in a given experiment € takes a particular value 
£,, which we call a realization (or sample value) of the random 
variable €. For example, consider the number of points which 
appear when a die is tossed, or the reading of a meter in a certain 
apparatus at a given instant of time. By repeating the experiment 
many times (and this possibility is implicit in the definition of a 
random variable), we can always obtain a large number of realiza- 
tions. Therefore, a random variable can also be characterized by a 
certain statistical ensemble of realizations. By subjecting the realiza- 
tions of a given random variable to statistical data processing, we 
can find certain of its statistical characteristics. Therefore, when we 
say that a random variable is “given,’’ we mean that we know the 
statistical data which completely characterize it. 

The simplest statistical characteristic of a random variable is its 
‘mean value: If &,, €,... are realizations of the random variable £, 
then the mean value of £ can be defined by 


‘ ’ 
t 2? 


Wong es £, + £2 +.» + &, 
3 (6 = fim Ae, 


(1-1) 
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i.e., <€> is the limit of the arithmetic mean of the sample values as 
the number of sample values (with respect to which the averaging is 
performed) is increased without limit. Probability theory can only 
be used to study experimental data for which such limits exist and 
do not depend on how the realizations &,, &,, ..., €, are chosen from 
the total statistical ensemble. Naturally, in making this choice, no 
information involving the specific values of the realizations should 
be used.! 

Other statistical characteristics of a random variable € can be 
defined by first going over to a new random variable y = f(€), and 
then using formula (1.1) to calculate <n); in so doing, we explicitly 
use the form of the function f(é). Thus, setting f(f) = &%, we 
define the mean square of by the formula 


2 2 2 
<@> = lim Sit o2 tt ba : (1.2) 


ta) 
nan n 


As another example, consider the probability P{€ << x} that the 
inequality € << x holds, where x is a fixed number. To calculate 
P{é < x}, we introduce the random variable 


1 for z>0, (1.3) 


Se eg, WERT NE) | heer 6 cig, 


Then we have 


P(E < x} = (He — £), 


1 We address the following remark to the reader who ts already acquainted 
with the subject: The approach based on the limit (1.1) is simple, intuitive 
and quite adequate for use in the applications; it has certain defects from a 
formal mathematical point of view, but these are unimportant when dealing 
with applications of probability theory. The point is that as far as the 
applications are concerned, the assertion that a given event has probability 
equal to unity (or arbitrarily close to unity) is equivalent to the assertion 
that the event occurs. Therefore, instead of the familiar requirement that 
the convergence expressed by (1.1) occurs with probability arbitrarily 
close to unity, we choose as our starting point the requirement that (1.1) 
actually converges. The reader will find other ways of dealing with this 
point in other treatments of the subject. 
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Le, 

P{é < x} = lim a ee (1.4) 
In fact, bearing in mind the definition of the function #(z), we see 
that the average of the arithmetic mean (e, + €, + ... + €,)/m IS 
just the ratio mjn, where m is the number of inequalities 
£,< x (1 <i <n) which are valid, and 7 is the total number of 
trials. Thus, (1.4) can be written in the familiar form 


| Plt <x} = lim. (1.5) 
nw n 


The mean value <# (x — &)> regarded as a function of x, i.e., 


F(x) = (x — £)), (1.6) 


is called the distribution function of £, and its derivative 
; ; | 
w(x) = = F(x) (Fc) =. J. w(x") dx’) (1.7) 


is called the probability density of £. Substituting (1.6) into (1.7), and 
formally differentiating under the averaging sign, we obtain 


w(x) = Xf — x), (1.8) 


where 
He) = EK 
2) = 7 Ae) 
is the Dirac delta function, which has the property that 


i” f(z) 8(@ — 29) dz = f(z) for any function f(z) . 


In what follows, we shall often be concerned with probability 
densities. For brevity, it is sometimes convenient to denote the 
value of x by the same letter as the random variable € itself, so 
that w,(€) becomes w,(£). We then drop the redundant subscript 
and write simply w(£) for the probability density of the random 
variable €. 
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The probability density w(£) satisfies the normalization condition? 


J me) de =1, (1.9) 
which, according to (1.8), is equivalent to the trivial identity 


<> =1. 


A random variable € is completely characterized by its probability 
density, and knowing this probability density, we can calculate the 
mean value </(€)> of any function f(€). In fact, using the basic 
property of the delta function, we can first write /(€) as 


f(€) = | le) BE — x) de 


and then calculate <¢ /(€)>. "Taking the average under the integration 
sign, we see at once that 
HED = | fle) BE — =) de, 


Or 
HED = | f(2) wx) de, (1.10) 


because of (1.8). 

Next, we show how the probability density behaves under 
transformations of the original random variable £. Let the new 
random variable 7 be defined by 


7 = a(f)s (1.11) 


where g(x) is a known function. Applying (1.8), we write the 
required probability density of the new random variable y in the form 


wy) = <&n — y)> = <d{e(é) — yy). 


2 When limits of integration are omitted, they will always be understood 
to be from —o too, 
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It then follows from (1.10) that 


wy) = J S{e(x) — ¥] ex) dx, (1.12) 


or more concisely, 
\ 


wo(n) = | Bfg(é) — a] mE) dé. (1.13) 


This last integral can be evaluated by using properties of the delta 
function. If the function (1.11) is monotone, it has a unique 
inverse function 


E=htn) (k=). (1.14) 
In this case, it follows froin (1.12) that 
| se HE) 2 sean) | 
mh agape MMM Sys 
; dé t=h(n) 


i 
In the general case where the function g(£) is not monotone, and 
instead has “several inverse functions” 


E=hf{n) (=1,2,.47), (1.16) 
formula (1.12) gives 
- 


wn) =D athiend | Z| (1.17) 


p8 tw g=hyln) 
ie 


| Besides the probability density w(g), the characteristic function 
Ou) = Cem) = | etMewlE) de (1.18) 


completely characterizes the random variable ¢. In fact, w(€) is 
just the Fourier transform of @,(u): 


w(é) = z J eit’ @(u) du. (1.19) 
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The moments 


m, = <€") (n = 1, 2,...) (1.20) 


of the random variable € can be obtained from the-characteristic 
function by differentiation 
1 d"Ou) 


my, = = — : 
"a8 dur yarn 


(1.21) 


From a knowledge of the moments m,, 1.e., the values of the 
derivatives of the characteristic function @,(u) at the origin, we 
can write ©,(u) as a Maclaurin series: 


Ou) = 14+ 5 m,. (1.22) 


nal a. 


Of course, we can only write (1.22) when the moments m, are 
finite and do not grow too rapidly as the index 7 is increased, 

To completely describe a random variable, one has to know all 
its Moments m,, m,,.... but most of the physically important 
information is contained in the first few moments. The higher 
moments describe rather unimportant details in the behavior of the 
probability density, which pertain mostly to extreme values of the 
random variable. For a variety of reasons, it is more convenient to 
characterize a random variable not by its moments, but by its 
cumulants (ot semi-invartants) k,, defined by the relation 


iu)" 
n!\ 


@,(u) = exp > kf (1.23) 


Comparing (1.22) and (1.23), we can obtain the following formulas 
relating the moments and the cumulants: 

k =m, 

k, = m, — mi, 
k, = my — 3mm, + 2m), a) 


ky = m, — 3m? — 4mm, + 12mim, — 6m}, 
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trom the structure of these formulas, it is clear that if we know the 
tirst nm moments, we can calculate the first 2 cumulants, and con- 
versely, i.e., the first 2 cumulants contain just as much information 
as the first 2 moments. If we confine ourselves to a definite number 
of first cumulants, it is convenient to set the higher cumulants equal 
to zero, To deal with moments in the same way, i.e., to set all 
moments higher than a certain order equal to zero, is not desirable, 
since then the characteristic function would behave at infinity in a 
way which makes it difficult to calculate the inversion integral (1.19). 
In fact, w(€) would then become a sum of derivatives of delta 
functions, with no simple intuitive meaning. 

The first cumulant is the same as the first moment, i.e., 
m, = k, = <>, and is just the mean value, the most important 
characteristic of a random variable. The next two cumulants are 
the central moments of orders two and three; 


ke=C(E—COP, k= CE—<O!. (1.25) 
Fhe quantity k, = <€> — <>? is called the variance (or dis- 
persion) of £, and is denoted by 
, 


Dé = <)> — «6. (1.26) 


The variance is also a very important characteristic of a random 
variable, since the smaller the variance, the more exact our 
knowledge of the random variable. The probability that the absolute 
value of the deviation of £ from its mean value <£> exceeds a fixed 
« > 0 approaches zero as Dé approaches zero, Therefore, if Dé 
is small enough, we can assume that € differs by very little from <>. 
- Instead of the variance, one often deals with the standard 
deviation 

o(f) = [Dé]? = [¢&> — (Hp. (1.27) 
While retaining the important features of the variance, the standard 


deviation has the advantage that it has the same physical dimension 
as the random variable itself. The standard deviation indicates the 
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order of magnitude of random deviations from the mean value, i.e., 
E— <£ ~a(€). (1.28) 


For the time being, we shall not discuss the significance of higher 
cumulants. However, we note that the ratios 


k k 
Ye = Pe Ye = ez (1.29) 


are called the coefficient of asymmetry and the coefficient of excess, 
respectively. The physical meaning of the cumulant &,, decreases as 
n increases. 


t 


2. Correlation Between Random Variables 


Suppose we have r random variables &,, ..., €,. Loosely speaking, 
this means that as a result of each experiment (trial) we simul- 
taneously obtain a realization of each of the r random variables. 
Vhe random variables €,,...,€, are completely characterized by 
tlie (joint) r-dimensional probability density 


Wy srg (Hyp sony Mp) = (O(E, — 24)». HE, — HD, (1.30) 


or simply w(é,, ..., €,), which is a function of r variables. It follows 
by definition that the mean value of any function /(é,, ..., €,) can 
be calculated by integrating with weight w(,,..., €,): 


CflEan ous Ee) = fame [A Erp oes &) Oban oon & 9) dE ow dE, (131) 

If we set f = 1, we obtain the normalization condition 
J S | w(&,, ..., £,) dé... dé, = 1. (1.32) 
Next, we divide the r random variables ¢,, ..., €, into two groups 


és, + €&, and ,4;,...,€,, and we ask whether knowing the actual 
realizations of the random variables €,,,,..., €, gives us any 
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information about &,,..., €,. For example, suppose we have two 
random variables £, and £,, described by a probability density 
w(é,, §2). In the absence of knowledge of the actual realization 
(i.e., pbserved value) of £,, the random variable €, has the prob- 
apt density 


- 081) = | (Ey, &) dee (1.33) 
\ 

Knowing the observed value of the random variable £, (which gives 
us complete information about £,), we can also obtain some infor- 
mation about the other random variable £,. Of course, in general 
our khowledge of é, is still imperfect, even when we know the 
observed value of €,, and hence £, remains undetermined, i.e., is 
still ai random variable. However, €, is now described by another 
probability density 


a= _ wl, £2) ae w(E,, 7) 34 
ce J (En &2) dé wf)” ee) 


which contains more (or at least not less) information about &, 
thari the distribution (1.33). The expression (1.34) is called the 
conditional probability density of &, given the value of &. The 
integral appearing in the denominator of the middle term in (1.34) 
guarantees that the normalization condition 


{més | 2) @e, = 1 (1.35) 


is satisfied. 

Just how much information is gained about £, by knowing the 
value of £, depends on the particular case. In some cases, no 
information about €, is gained, regardless of the observed value of 
£,, which means that 


oo w(E, | &,) = w() (1.36) 


i- 


3 Asis pustomary, probability densities will often be referred to (somewhat 
imprecisely) as ‘‘distributions.” 
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We see from (1.34) that in this case w(£,, &) is just the product of 
the two one-dimensional probability densities w(£,) and w(£,), 1.¢., 


w(Es, $2) = w(E,) w(E2) , (1.37) 


and the random variables £, and £, are said to be (statistically) 
independent. The opposite extreme is the case where knowledge of 
€, completely determines the other random variable ¢,, which is a 
function of £, [, = f(&,)]. Then, €, and £ are said to be completely 
correlated, and their joint distribution contains a delta function: 


w(f1, 2) = [, — f(é2)] w(Ee) - (1.38) 


Naturally, there is a whole range of intermediate cases between 
these two extreme cases. 

All that has been said above about two random variables can 
immediately be generalized to the case of several random variables. 
Given 7 random variables £,, ..., €,, knowledge of several of them, 
€.g., Enity «+ €, Increases (or at least does not decrease) our infor- 
mation about the other random variables £,, ..., £&,. We can express 
the conditional probability density of £),...,€, in terms of the 
original probability density w(&, ..., €,) as follows: 


WE, -+ Ei | Steaas sees £,) = aE ‘ (1.39) 
If knowledge of &;41, ..., §, does not increase our information about 
£,, -- €, then the two groups of random variables ¢,, ..., &, and 
Exit --y &, are said to be (statistically) independent. However, if the 
conditional distribution (1.39) does not coincide with the ‘‘uncon- 
ditional” distribution 


w0E gy ony Ga) = fone f 00(Eay voy br) dbesa on Ey, (1.40) 


then there is statistical dependence between the two groups of 
random variables. 

It is now convenient to introduce numerical characteristics which 
describe the degree of statistical dependence between random 
variables, and which vanish when there is no statistical dependence 
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bezween them. Such characteristics are provided by multiple 
cotrelations of various orders. The covariance (synonymously, cross 
correlation or double correlation) of two random variables £, and &,, 
denoted by K[é,, €,], is defined by the relation 


K(é,, Es] oa (1&2) = <Ep> «&> . (1.41) 


If £, and £ are independent, we have 


(fifa = CE (fe). 


according to (1.37). Hence, in this case, the covariance vanishes. 
By definition, the covariance of a random variable with itself is just 


its variance 
n K{é, é] = Dé * (1.42) 


Sometimes, instead of K[£,, €,], it is convenient to consider the 
dimensionless quantity 


Ml 


i — Kifer f2) 
, aie (TCA Soe 


which is proportional to K[é,, €,]. The quantity (1.43) reduces to 
uhity when the random variables coincide, and is called the 
chrrelation coefficient of £, and &,. 

| Besides double correlations, there are multiple correlations of 
higher orders. Thus, given three random variables £,, £, &, 
we can construct a triple correlation K[£,, £, £,), which describes 
statistical dependence involving all three random variables £,, £2, &5, 
and vanishes when at least one of the random variables is inde- 
pendent of the others. We first form the quantity 

{ 


(E,b4&3> = <é> <2) <3) » (1.44) 


by subtracting the product of the mean values of the random 
variables from their joint third moment. The quantity (1.44) has 
the property that it vanishes when all three random variables are 
independent of each other; however, (1.44) can be different from 
zero not only because of correlations involving all three random 
vaniables, but also because of correlations between pairs of random 
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variables. For example, suppose that one of the random variables, 
€, say, is independent of the other two. Then, there is no triple 
correlation, but the quantity (1.44) can be different from zero 
because of the presence of double correlation between £, and &,. 
In fact, in this case, we have (£,£.£,> = <£:£2> (€3>, and (1.44) 


equals 
<€3> K{é1, &4] - (1.45) 


Thus, it is clear that to properly define the triple correlation 
K[é,, €2, £3], we have to subtract from (1.44) all terms of the form 
(1.45) due to the presence of double correlations. There are three 
distinct pairs which can be selected from &,, &2, &3, 1.€., 


£1, fo3 £5, 3; £1, &3. 


Therefore, subtracting all terms involving double correlations from 
(1.44), we define the triple correlation K[£,, £,, £3) by the formula 


K{é1, fa fa] = <f1babs> — (E> K{fo, &3] — <f2> Kf, £5) (1.46) 
r= (fy) K{é,, E,] a, <é> <2) <&3> . 


Using (1.41), we can write (1.46) in the form 


K{é,, £2, 3] = <&1Saks> — ¢£1> ¢f2bs> — <E2> (Eiba> 
— CEs) (£1 f2> + 2¢£1 (Ee) CEa> - 


In the case where all three random variables coincide, the triple 
correlations (1.47) reduces to the third cumulant 


K{é, é, 6] = hy. (1.48) 


Moreover, by analogy with (1.43), we can introduce the triple 
correlation coefficient 


R, —s K(é,, be, £3] {K{é,, 1, £1] K{é,, £., £4] K{é;, £3, é,)}-14 . (1.49) 


To define the fourth-order correlation for four random variables, 
we have to subtract all possible double and triple correlations from 
the difference 


(1.47) 


(i bobsb4> - <&> <> <é) (Ea : (1.50) 
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The number of terms to be subtracted is determined by the number 
of possible combinations of the elements £,, €,, £5, &, arranged in 
groups of one, two or three elements. The formulas for higher- 
ordér correlations become more complicated as the order is 
increased. 

The most concise expression for mean values of products of 
sevéral random variables can be written in terms of the charac- 
teristic function 


O(uy, .) Up) = Cexp tue, + ... + Uy E,)> « (1.51) 


In fact, we have 
i] 


Chic Pies oh CO a) 


i” Oty. Ou, 


(1.52) 


uy™. . o™U,y=0 


To obtain the corresponding formula for the multiple correlation 
of ofder 7, we need only replace the characteristic function by its 
logarithm in (1.52): 

, 1 


rl ag 
Flas Go] = Ge ted / (153) 


uy=,,.=Uy=O0 


3. Random Functions 


Next, we consider a random function £(t) of a single real argument 
t (che time) which varies over the interval [0, 7], say. The random 
function &(t) will be regarded as given if we know stable experi- 
mental conditions which allow us to obtain realizations of £(¢) in the 
interval [0, 7]. For example, such a realization might be a phofo- 
graph (or oscillogram) of the random function, whose form depends 
on time. 

If we take 7 fixed values ¢,, ... t, from the interval [0, 7], where 7, 
t,,...,¢, are arbitrary, then the values é(t,), ..., &(¢,) constitute a 
family of random variables. If the random function is completely 
specified, these random variables are also completely specified, 
for arbitrary 7, t,, ..., ¢,. Therefore, we can regard the corresponding 
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probability density w[£(t,), -.., &(t,)] as specified. This probability 


density is a function of 7, ¢), ..., ¢,, and can be written as 
CBfxc, — E(t,)] os. Bx, — E(t ID = 7 4(Xqy one Kp Ey ony te) (LSA) 


Each probability density (1.54), regarded as a function of x, «.., *,, 
satisfies the usual requirements imposed on probability densities, 
i.e., nonnegativity and the normalization condition. At the same 
time, as functions of ¢,,...,¢,, the probability densities (1.54) 
satisfy certain additional conditions, which are a consequence of its 
definition. These are the symmetry condition 


We (si Xk oey xt, ney oaey te, eey thy «03) — Wel ws, XI, eoty Xres aeey oeey ty eee, ty sve) 
and the compatibility condition 


Wel, -n0y Ky} by, o02, ty) = f erase say Regs bay sony Epp ey OX yp gy one Uhh yp + 


(1.56) 


We can consider a random function £(t) to be specified if the prob- 
ability densities (1.54) are specified for all 7, no matter how large. 
By choosing the points of division ¢,, f,...,¢, sufficiently close 
together, we can replace £(t) by a sequence of random variables 
&(t,), €(t2), ..., €(t,), with an accuracy which is sufficient for all 
practical purposes. Thus, a random function £&(t) can be charac- 
terized by an infinite sequence of probability densities 


w(x; t), Woy, X25 ty, fe), vey (1.57) 


which satisfy all the necessary requirements. 

If we know the first 7 probability densities, we can discard the 
probability densities w,,...,%,_1, since they contain no extra 
information not already contained in the probability density 
W(X, ++, p53 ty, ..., #,). In this regard, it should be noted that the 
ideal state of affairs would be to describe a random process by a 
single probability density of the largest possible order, if such 
existed, but since the argument ¢ is continuous, there is no 
probability density of maximum finite order. However, it is 
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V4 
dotnelimes possible to introduce a probability functional W[E(t)], 
which defines a “continuous probability density,” i.e., defines a 
probability density in an appropriate function space. 

Instead of the probability densities (1.57), we can give the 
foliowing sequence of characteristic functions: 


@, (1; t,) = Cemrele) , 
ve (1.58) 
Oo(uy, yj ty, te) = Cettrkltyd+ingklta» , 


Specifying (1.58) is equivalent to specifying the sequence of 
distributions (1.57). Here, as before, we need only retain the 
characteristic function of highest order (if there is one), since all 
characteristic functions of lower orders can be obtained from the 
one of highest order by setting certain arguments equal to zero: 


0,(41, seep Uy ti, avey ts) — Or4 x(t, woop Uy 0, ovey 0; ti, eey ty sk)s (1.59) 


By increasing the order 7, thereby approaching the ‘‘most complete 
‘sHaracteristic function,” containing as much information as 
pessible about £&(¢), we obtain in the limit the characteristic 
junctional 


bs Ofu(t)] = (exp [i { w(e) g(t) dt} ) (1.60) 


which completely describes the random process £(¢). In this 
formula, the integral in the exponential extends over the whole 
‘demain of definition of the random function. The characteristic 
¥unctions (1.58) can be obtained from (1.60) by choosing argument 


‘functions of the form 
i ty 


u(t) =D) u,5(t — t,) « (1.61) 


{ 

In its general form, a random process‘ cannot be described by a 
fikite number of functions of a finite number of variables, and 
hence a random process is either characterized by an infinite 


‘The term random process (or simply process) is used as a synonym for a 
raadom function of time. 
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sequence of functions or by a functional. Instead of the sequences 

(1.57) and (1.58), itis preferable to consider a sequence of functions 

such that the functions of higher order do not repeat the informa- 

tion contained in the preceding functions, but instead carry only 

new information. Therefore, we now introduce another method of 

describing random processes, which has this and other advantages. 
The mean values 


CE (Ey) 2 Ele) > = Mr(tay wy by) (1.62) 


regarded as functions of ¢),.,., ¢,, are called moment functions, and 
the infinite sequence of moment functions 


my(t;),  — myg(ty, to), Mig (ty, tay tg)» oe (1.63) 


gives an exhaustive description of the random process £(2). It is 
even better to characterize the random process £(t) by the sequence 
of correlation functions 


Ry(t)), Realty, te), Agta, ter ta), os (1.64) 


which are defined as the multiple correlations 


R,(t,, .., tp) = K(E(2,,) ..., €¢,)] , (1,65) 


regarded as functions of the times ¢,...,¢,. The correlation 
functions, just like the moment functions, are symmetric functions 
of their arguments. As the order increases, the physical significance 
of multiple correlations decreases, and hence the first few functions 
in the sequence (1.64) are the most important. This is the advantage 
of describing a random process by its correlation functions, The 
vast majority of random processes encountered in radio physics 
have the property that when the times ¢,, ..., ¢, are moved apart, 
the correlation between the corresponding values of the process 
falls off, i,e., the correlation functions (1.65) go to zero (for z > 1), 
which is very convenient. 

If we know the moment functions or the correlation functions, 
we can find the other characteristics of a random process, in 
particular, its characteristic functions or probability densities. In 
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; se 

fact, using (1.52) we can write the characteristic function as a 
multidimensional Taylor’s series: 

O,(u1, ooey Ups Ly cory t,) ars l Bs LF ste > mete moa t.) Ug. Uy. (1.66) 
a 

|n just the same way, using (1.53), we can express the charac- 
teristic function in terms of the correlation functions: 


F oO r 
(ty, 0 Ups by, oop Ep) = EXP > sr > Reg(tay one) Ly) Ug oe Uy) » (1.67) 
s=1 Ap. Wal 


It is important to note that (1.67) does not change its form if we 
choose different times ¢), ..., ¢, or change the number r. Using this 
fact, we can write the characteristic functional (1.60) as 


‘O{u(z)] = exp »2 f | ake J Aalty wey t,) u(t,) ... u(t,) dt, ... dts). (1.68) 


Tn Chap. 3, we shall show how to go from characteristic functions 
lo probability densities. 

' Because of the special role played by moment functions and 
correlation functions, we give the following formulas relating 
‘them: 


my(t,) = k,(t1), 
me (ty, t2) = Re(ty, te) + Ry(ty) Ay(te) 
mg(ty, te, ty) = Ra(ty, tey ty) + 3{Ry (tr) Kalter ts)be + Rata) Malte) Ay(ts) 
Mids toy tay £4) = Rg(ts, tar tar tg) + 3{Ro(t,, te) Rolts, tad} (1.69) 
+ ALR, (t,) Baltes tas t4)}s + O{Rr(t1) Ry(te) Raltar ts)}s 
+ y(t) Ry (te) Ri(ts) Aaa) » 


ene ee a ee 


i Here, the symbol {...}, denotes the operation of symmetrizing the 
expression in brackets with respect to all its arguments. If the 
‘function to be symmetrized depends on m arguments and has no 
symmetry at all, this means that we have to carry out all 2! possible 
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permutations of the arguments and then take the arithmetic mean 
of the resulting 7! terms. However, if the function has symmetry 
with respect to some of its arguments, then among these z! terms, 
there will be some that are identical. Therefore, we need not 
calculate the arithmetic mean of all the permuted terms, but only 
of those which are different. It is interesting to note that the 
coefficients appearing before each symmetrized expression in the 
formulas (1.69) are equal to the total number of permuted terms 
which are not identically equal. To form the arithmetic mean of the 
terms which are not identically equal, we have to divide the sum 
of these terms by a number which just equals the factor standing in 
front of the sum. Thus, all that remains in (1.69) are sums over 
those permutations of the arguments which lead to terms that do 
not coincide. 

The system (1.69) can be solved step by step for the correlation 
functions in terms of the moment functions. It should be noted 
that the correlation functions are not the same as the central 
moments. The difference begins with k,. Thus, if we set k, = 0 
in (1.69), we find that m,, the central moment of order four, differs 
from k, by the term 


Mg (try tay bay t4) — Rgltys toy ty, ta) = Reo(ty te) Reoltar 44) (1.70) 


+ R(t , ty) Ro(tor ty) 4- Ro(ty, ta) Kalter ta) - 


\ 
| 


CHAPTER 2 


Stationary Random Processes 
and Spectral Densities 


1. Basic Concepts 


' A random process is said to be stationary (in the strict sense) if its 
statistical characteristics are invariant under time shifts, i.e., if 
they remain the same when ¢ is replaced by ¢ + a, where a is 
arbitrary, Then, the probability densities w,(&, ..., Enj ty, «+» tn); 
tégether with the moment and correlation functions m,(¢;, ..., tn) 
and k,(t,, ....¢,) do not depend on the absolute position of the 
points ¢,,...,¢, on the time axis, but only on their relative con- 
figuration, i.e., only on the time differences ¢, — t,..., t, — t. 
For example, consider the correlation function &,, (¢,, ..., t,). By the 
definition of stationarity, we have 


R(t, oy ta) = Rat, + 0, wr tr + @). (2.1) 
Choosing a = —t¢, and writing 
k,,(0, te eas ti eacy b = t1) = Ri(te a ti oeep t,, = ti) > 
we find that 
Raltss eoey t,.) = Ri(te — ty eoey tn oa t,) . (2.2) 


‘Thus, the correlation functions of a stationary process depend 
only on the time differences, as asserted. In particular, it follows 
that the mean value k,(t,) of a stationary process [equal to &,(0)] 
must be a constant, which we denote by m: 


pany Sm: (2.3) 
21 
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Similarly, the correlation function &,(t,, ¢.) is a function of t, — ¢,, 
for which we introduce the special notation 


R(t, te) = R(t, = ty) ’ (2.4) 
where k(r) = k(—71). We can also write A(r) in the form 
k(r) = o*R(r), (2.5) 


where o = o[€(t)] = V R(0) is the standard deviation of &(t), and 
R(r) = R(r)/o? is the dimensionless, normalized correlation 
coefficient (1.43), with the property that R(Q) = 1. We shall not 
introduce any special notation for the higher-order correlation 
functions, since they are not as important. It should be noted that 
when all the arguments of the third-order and fourth-order 
correlation functions coincide, we obtain the cumulants &, and &,, 
appearing in the definition (1.29) of the coefficients of asymmetry 
and excess: 


Ya = o-k,(0, 0) , ye. = a *k,(0, 0,0). (2.6) 


In what follows, we shall often use the concept of the correlation 
time, defined by the relation 


Beet [1 Re) ee fae) ae (2.7) 


The correlation time tegr gives some idea of the size of the time 
interval over which correlation extends between values of the 
process &(t). In fact, we can neglect correlation between values of 
the process whose time separation is considerably greater than 
T-sr- We also define the quantity 


K=[ Wr)dr, (2.8) 
which we call the mtensity coefficient of the random process £(¢). 


This quantity plays an important role in the theory of Markov 
processes. 
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« Next, we introduce the important concept of the (power) spectral 
density S[£; w] of a stationary random process £(t): 


5 SE; w] = 2 {: etortt Y dr = 4 if cos wri ff, dr. (2.9) 


Here, and subsequently, the subscript + on a function like £,, in 
the presence of a similar function like € without a subscript, denotes 
h shift of the argument by an amount 7, i.e., £, = &(¢ + 7), while 
¢ = &(t). For a process with zero mean value, S[§;w] is the 
Fourier transform of the correlation function: 

. S[E w] = 2 is ef" k(r) dr. (2.10) 


In the general case, where the mean value ¢£> = m Is not zero, a 
similar formula holds after subtracting m from &: 


S{é — m; w|) = 2u(w) = 2f- e'% R(r) dr. (2.11) 


Then, the spectral density S[£; #] contains a delta function term at 
ero frequency. In fact, substituting <€€,5 = A(r) + m?® into (2.9), 
we obtain 

SE; w] = S[E — m; w] + 4m 5(w) . (2.12) 


It follows from (2.8) and (2.11) that the relation between the 
intensity coefficient and the spectral density is 


K =} S{é—m;0]. (2.13) 


_ If we take the inverse transform of (2.11), we find that 


1? 
Kr) = =—f eit SLE — mw] dn ae 
= $e J cos or Se m; w] dw, 


) « 
In particular, this gives the formula 
1 


i o*%(£) = A(0) = = a S[E — m; w] dw (2.15) 


! 
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for the variance of &. It can be shown that the spectral density is 

nonnegative for all w, and hence can be interpreted physically as the 

distribution over frequency of the power of the process £(¢). 
Inverting (2.9), we obtain 


(fi) = a { cos wr S{E; w] dw. (2.16) 


Sometimes it is convenient when calculating spectral densities to 
use techniques of operational calculus and the formula 


S[é w] = 4ReL[<éé,); tw] , (2.17) 


which follows from (2.9); here, L[f; p] denotes the Laplace trans- 
form 


Ufal= fer seae. 


Example 1. Let &(¢) be an exponentially correlated random 
process, with correlation function 


k(r) = o%e-Att, (2.18) 


Then, according to formulas (2.11) and (2.7), &(t) has the spectral 
density 


40” 
Slé — <3 0] = SF, 
w + fP 
(2.19) 
and the correlation time 
l 
Toor = (2.20) 
Example 2. If the correlation function has the form 
k(r) = o%e-Fl"! cos war , (2.21) 
the spectral density equals 
| cc atitiae. w+ we + p? 
S{é = <8); w] ran 480? (w? — w? — BY? + apa? . (2.22) 
If the inequality 
B < Wo (2.23) 


holds, the spectral density ts concentrated in a relatively narrow 
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frequency band w — wy ~f, and we have the approximations! 

1e— Dig w Hor tek et 

S{E C65 w] oy 480? (w? ee w?)2 a 4Bew? og (@ a wa)? + pe , 
2 1 


owt ‘ 
Tear Pe Bg 


' The following table exhibits these and other commonly encoun- 
tered correlation functions, together with the corresponding 
I nee 

spectral densities: 


TABLE [| 

' No. k(r) SLE — m; w] = 2x(w) 

1 K8(r) 2K 
a | aaa: 4B 2 
ve ei aR 

3 o%e—1/4p%x Vm en" iB? 
-+-———— 
5 2 2 a 
t -Alr wo + wy + B 
! 4 ate Bl | COS wot 4Bo® (w = we — a3 + 4B? w? 


p . wo + f° 
Blt} w _— w 722. 2 A? LARD D 
15 [eA (c08 aor + Pies )| 4 (w? — 2 — p%)* + 4p?u2 


w* 


1 6 e~lBlt (cos wot — £ Sin [ wo7 } 4p - 


; (at at Py aot 
7 a feral") — oe-Fil AaB + A) 08 
B-a (w? + a7) (w? + 8?) 
3 of sin Br Daves 


ifw < £;0if w > B 
Br | B 
The symbol ~ means ‘‘is approximately equal to,’ while ~ means ‘‘is 
of the order of.” 
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2. Calculation of the Spectral Density 
by Using the Laplace Transform 


Consider the Laplace transform of a stationary random process 


E(t): 
LEP] = |e? et) dt. (2.25) 


Multiplying this expression by its complex conjugate and averaging, 
we obtain 


CIEE PIED = ff ete wt Ce) at')y at at’, 
0 0 
where the asterisk denotes the complex conjugate. Making the 


change of variables 


we find that 


CUE PIR =f arf ele +p pl Cee) ds. 


Integrating with respect to s, we obtain 


l 


<I LE; p] |? = Same l: 


oo — f* 

[__exp}—(e +995) PSP cee par, 
Multiplying by p -+ p* and comparing the result with (2.9), we 
arrive at the formula 


S(E; w] = 2 lim (p + p*) <I L(A], (2.26) 


which is sometimes useful for calculating spectral densities. 

Since the stationary process £(¢) does not grow at infinity, the 
Laplace transform (2.25) is defined for Re p > 0, and so is the 
quantity < | L[g; p] | 2). If we set 


SEC. 3] STATIONARY PROCESSES AND SPECTRAL DENSITIES 27 


then the limit p —- —zw corresponds to the limit « + 0, Therefore, 
instead of (2.26), we can write 


S{é; wv] = 2 lim zd 


Uf 


> (2.27) 


3. The Random Spectrum and its Relation 
to the Spectral Density 


The spectrum y(w) of the process &(¢) is defined by the formula? 


wv) = Te fens e(eyat, (2.28) 
and is obviously a random function of the frequency w. 

* Strictly speaking, when &(t) is a stationary random process, the 
integral (2.28) diverges, so that (2.28) has only a formal meaning. 
In fact, (2.28) must then be interpreted as meaning that an ordinary 
equality is obtained if we apply a linear integral operator J, to 
both sides of (2.28), and then interchange the order of integration 
in the right-hand side: 


! ? —2 art 
Joyo) = Tae J Jue") a) dt (2.29) 


Actually, this is the same way in which the familiar formal relation 
iy eee 
(r) = = | en dy (2.30) 


is interpreted. The expressions (2.28) and (2.30) are so-called 
generalized functions. Such a function is regarded as specified if we 
know the integral of the product of the function with an arbitrary 
continuous function. Moreover, it is not necessary that the value of 
a generalized function be specified for every value of its argument. 
As is well known, one can deal with generalized functions like (2.28) 
and (2.30) in the same way as with ordinary functions. x 


2The terms spectrum and spectral density are sometimes used as (rough) 
synonyms in the literature, but here their meanings are entirely different. 
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Multiplying (2.28) by its complex conjugate and averaging the 
result, we find that 


(Hw) ¥*(@')> = a { | __ ertwttiot (Et) E(t) dé dt’. (2.31) 
Making the change of variables 


r=? -1?, p= att), 


or equivalently 
T ; T 
t=b—->, Pats: 


and then integrating with respect to t), we obtain 


Cola) yX(a')y = Ho —a') [exp i 2 $5] Ce,) dr, 


where we have used (2.30). According to (2.9), the remaining 
integral is just one half the spectral density S[£; w], and hence 


Cola) y*(w')y = 5 80 — a") SIE; 0] (2.32) 


The inverse of the transformation (2.28) is 
Hie i gli Sch tah (2.33) 
V 2n ~o 


Since the process £&(t) is real, taking the complex conjugate of 
(2.28), we easily obtain 


y*(w) = y(—4). (2.34) 


We can then write (2.33) in the form 


&(t) = fe Re f ; get ia)dea, (2.35) 


which is the customary way of writing oscillations in radio 
engineering (the method of complex amplitudes). Formula (2.35) 
represents the process &(t) as a superposition of harmonic oscilla- 
tions, where, in terms of the complex spectrum y(w), the oscillation 
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of frequency w has amplitude proportional to | y(w) | and initial 
phase equal to arg y(w). 

*x The random spectrum y(w) appears only in the integrand of 
(2.35). Thus, to use this formula, relations of the type (2.29) are 
sufficient, and the strict equality (2.28) is not necessary. By the 
same token, to make our treatment more rigorous, we ought not to 
require that (2.31), (2.32) and (2.34) hold, but only that the corres- 
ponding formulas obtained by applying the integral operator J, 
hold. The use of integral operators is also justified physically, since 
experimentally we cannot obtain an exact harmonic, but can only 
isolate a sum of harmonics lying in a finite (albeit narrow) fre- 
quency band 4w. 

Choosing j,, to be the operation of integrating with respect to w 
from w, to wo + Jw, with weight (1/4w)e's, and writing 


] otdw : 
Y(wp, to) = Aa f eo y(w) dw , 
m9 


we obtain 
| — e-i4uxt—ty) 


| a 
Y(wo, ty) = Vin { -@ idw(t — t%) 


e~rmoll-to) E(t) dt, — (2.36) 
according to (2.29). Carrying out similar integrations with respect 
to w and w’ in both sides of (2.32), we obtain 


133) 


<| Yoo to) 2> = Far _ S[E; w] dw . 


[f dw ts so small that the spectral density does not vary much in the 
interval from w, to w, + Mw, this formula reduces to 


C1 ¥ (eo te) P> = Ga SIE on 2.37) 


The formula (2.37) shows the connection between the spectral 
density and the function Y(w, t,), which to a certain approximation 
replaces the true spectrum y(w,)e‘“e. If the bandwidth dw is 
small, the factor 

| — en t4ate—ty) 


tMuxXt — to) 
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in (2.36) is slowly varying. This factor has the effect of restricting 
the region of integration in the divergent integral (2.28), and as a 
result, the influence of values of ¢ for which |t — t)|> 1!/dw 
becomes negligibly small. Of course, there are many other ways of 
restricting the region of integration in (2.28), corresponding to 
other forms of the integral operator J,, in (2.29). * 

Under actual conditions, one does not deal with the exact 
spectrum y(w). Thus, consider the function 


¥(u, 4) = f ; G(ty — 1) &(t) dt , (2.38) 


which is a good approximation to the spectrum y(w) if the function 
G(t) — 1) is “narrow-band.” We shall regard the function G(t, — 1) 
as being narrow-band if in the spectral expansion 


Cll — 1) = [ciate gu)dw —(f gw)dw = 1), 2.39) 


the function g(w) is appreciably different from zero only in a small 
interval 


| w —w| ~ Mw (Aw < wy) (2.40) 
centered at wo. This means that G(t, — #) is just the harmonic 
eol'o-) multiplied by a slowly varying function, which changes in 


a time of order | /4w. Substituting (2.39) into (2.38), changing the 
order of integration and taking account of (2.28), we have 


¥ (Wg, fa) = J ete g(w) y(w) deo. 
Multiplying (2.32) by g(w)g*(w’ Je?! and integrating, we obtain 
C1 ¥(m, te) > = 3 | SLE; @] | ew) |? dw. 


Suppose that the function g(w) is appreciably different from zero 
only in an interval (2.40) which is so narrow that S[£; w] practically 
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does not change there. Then, we can bring the function S[&; w] in 
front of the integral sign, obtaining 


| ¥ (Wo, t0) [> = $S[E eg] f | g(w) [2 do (2.41) 


_ According to formula (2.41), to obtain the spectral density it is 
necessary (in radio engineering language) to pass the random 
function &(¢) through a narrow band linear filter followed by a 
quadratic detector, and then average the result. However, in this 
regard, it should be pointed out that any actual radio engineering 
device has a real impulse response G(t) — t). This means that g(w) 
has the property 


g(—) = g*(w) ’ 


so that g(w) must be different from zero not only in the interval 
(2.40) but also in the symmetric interval | w + wy|~ 4w, where, 
because of its evenness, the spectral density S[£; w] takes the same 
values as in (2.40). Therefore, as before, we can bring S[£; w] in 
front of the integral sign. Thus, in the case of a real narrow-band 
process Y(wo, to), formula (2.41) remains the same. 


4. Time Averages of Stationary Processes 


, The so-called ergodic processes are an important subclass of the 
class of stationary processes. If, we know a single realization of 
an ergodic process over the whole time axis —~ <t < ©, then by 
thaking time shifts, we can obtain an infinite statistical ensemble of 
realizations. Therefore, from one realization of an ergodic process, 
we can ascertain all of its statistical characteristics. 

For example, the mean value of &(t) can be calculated by the 
formula 


. | ¢t 
m = im ie E(t) dt . (2.42) 
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Of course, other formulas can also be used, in particular, the 
formula 


1 —% 
m = lim — &(t + kd ; 2.43 
si, ay 2 £ ) (2.43) 


which replaces the integral (2.42) by a sum, where 4 is any positive 
number. To prove (2.42), consider the random variable 


mr = f : é(t) dt , (2.44) 


which has mean value <m,)> = m. As we shall now show, the 
variance Dim, goes to zero as T —> ~. 

Squaring (2.44), averaging the result and subtracting m?, we 
obtain 


I 4b or : 
Dmr = Fa | [Me —t) deat ° 
Making the change of variables 


r=t—t, t= $(t+t'), 


we see that for a fixed value of 7, the limits of integration with 
respect to ¢) go from $|7|to T — $|7|. Thus, integrating first 
with respect to ¢) and then with respect to 7, we obtain 


1 T 
Dm r= Fe f _ (TI) Re) dr (2.45) 


or because of (2.5), 


20? 


Dm = “2 fi Zs 5) R(r) dr. 


Since Dm, is nonnegative, it follows that 


Dmr = 7 | (1 aie R(r) dr < 2m “( ~ $F) | RG) | dr 


207 


Tr 207 
<p J I RO) dr <A] RG) ar. 
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The last integral is the correlation time (2.7), and hence 
’ 
Dmp < 20? —. (2.46) 


From (2.46) it is clear that if the stationary process &(t) has a 
finite correlation time and a finite variance o? = <€%(t)> — m?’, then 
‘ Dm7y—0 as Too, 


“ 


r 

This means that as T increases, the random variable m, converges 
to a quantity which is not random, i.e., the mean value m. The 
formula (2.46) also leads to the following rough estimate of the 
speed of convergence: 


| F if E(t) dt — m | <0 (2 Fae" (2.47) 


The time average (2.44) converges at the same rate as the arithmetic 
njean of a family of N identically distributed, independent random 
variables &(t,), where N = T/2r,,,. In fact, in order to simplify 
practical calculations of the mean value, it is convenient to go from 
the integral (2.42) to the sum (2.43) by choosing 4 = 27,6,. 

' Just as in the case of the mean value, we can use time averaging to 
calculate other statistical characteristics of ergodic processes. For 
example, forming the new stationary random function 


n(t) = &(#) E(t + 7) (2.48) 
(where 7 is fixed), and applying formula (2.42) to y(t), we find that 


ee a 
(E62) = fim — [, ee +7) dt, (2.49) 


a result which can be used when 7(t) has a finite correlation time. 

If we want to calculate the spectral density, then, according to 
(2,41), we have to tume-average the square of the function Y(w, tg). 
Both the process Y(wo, to) and its square are stationary processes, 
regarded as functions of the time zg. Since Y(w9, tg) is more narrow- 
band than the original process &(¢), it has a longer correlation time, 
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whose order of magnitude is inversely proporttonal to the band- 
width Mw. It is natural to assume that | Y(wo, to) |? has approxima- 
tely the same correlation time: 


(Tem )y? | /4w. 


Applying formula (2.47) to the process | Y(wg, to) |?, we have 


; l T IDI Y |? 1/2 
[¥en PY — pf 1X Pate] < (Hy) ~ 250) 
or taking account of (2.41), 
“ 2 I ¢* o(| Y |?) ! 
ieee tes el eae IE des | ee 
| [£5 ao] Sle Pde mie (Wo, t) |? deg Sle Pde V The 
It can be shown that the estimate 
a(| ¥ |?) 
+t wm Sfé; 
flgtde ~ 
holds. This implies that 
2 1 rf 
SU ape ae as Ee Ft 2.5 
ical orgs te Ce 251) 


with a relative error of order (T4w)-!/2, We obtain the exact value of 
the spectral density if we pass to the limits T4w —> © and dw —+ 0 
in (2.51). 

In making experimental measurements of the spectral density, 
it is not desirable to use a very small bandwidth 4w, since to obtain 
satisfactory accuracy the averaging time 7’ has to be very much 
greater than the time | /4w required for the narrow-band process to 
change appreciably. Of course, the operation of time averaging 
need not take the explicit form (1/T) J, ... dt. In fact, as a rule, the 
integrators which perform the time averaging in practice have other 
weight functions. The important thing is that the integration time 
T should be large enough to satisfy the inequalities 


T > Tears T >. 
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Suppose we are given several random functions £(t), y(t), ..., 
which for full generality, we assume to be complex. If the processes 
are stationary, then their correlation functions 

l 


K( E(t), F(t), K{n(4s), 0 * (ta) (2.52) 


depend only on the time difference ¢, — t,. In addition to the 
-orrelation functions (2.52), we are interested in cross correlations 
between different random functions, e.g., £(t) and (t). Two 
Stationary processes £(t) and 7(t) are said to be stationarily correlated 
if the function 

: Kf é(t1), 9 *(te)) = Reyfte — 41) (2.53) 


also depends only on the time difference t, — t;. The function 
(2.53) ts called the cross-correlation function, as opposed to the 
functions (2.52), which are sometimes called autocorrelation 
functions. Unlike an autocorrelation function, a cross-correlation 
function need not be an even function of its argument #, — t,. 

If the mean values <£> and <y> equal zero, then the Fourter 
transform of (2.53) gives the cross-spectral density 


S{E no] = 2 f , eoren*) dr, (2.54) 

which is not necessarily real, but has the property 
S*(E,n3 ow] = S{y, & o). (2.55) 

Moreover, for real processes, we have the relation 
S*{E,n, 0] = S[E,43 —o]. (2.56) 


If we set y = € in (2.54), we obtain the spectral density of a single 
random function. In the notation of (2.54), the spectral density 
S[&; w] defined by formula (2.9) can be written as S[&, &; w]. The 
inverse of the transformation (2.54) is 


en =r fo Se mel eter de. (2.57) 
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We now consider the random spectra 


rie) = ef” emands 
(2.58) 
Pa ee [eto a(t) at 
” A/D eee y ’ 
in terms of which we can write the processes £(t) and y(t) as 
superpositions of harmonic oscillations, by using expressions like 
(2.33), ort ltke (2.35) in the case of real processes. By the same 
argument as used to derive formula (2.32), we can easily obtain the 
formula 


Cw) ¥M(w')> = ¥8(w — w') SE, 95 &] (2.59) 


(rom (2.58) and (2.54). Next, we introduce the narrow-band random 
processes 


V (oq, ty) = [ eo ge() yew) deo = [ Ge(ty — 1) E(t) dt, 
(2.60) 
V (egy to) = | ee gq) (oo) do = [ Golly — 2) n(t) at, 


which, like (2.38), replace the exact spectra. Here, the functions 


ber Gas Bys G, have the same meaning as before, but in the general 


case, they may not be the same for the processes £ and 7. Using 


(2.60) and (2.59), we easily find that 
(¥e(wor bo) ¥A(wo, 4) = 4 f S[E, no] ge(o) g8(w) do. (2.61) 


The presence of the functions g,, g, in (2.60) converts. £, 7 into 
the narrow-band signals Y,, Y,. If g, and g, have the form 


E(w) = 8(%) = 8) , (2.62) 


where g(w) is a real function which is appreciably different from 
zero only in a narrow frequency band w — w, ~ 4w, then 


CY (9, to) Y* (wos to)> = 4 S{E, 73 wal xc dw. (2.63) 
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Therefore, to obtain the (complex) value of the cross-spectral 
density, we have to take the average of the complex quantity 


V,Y* =/¥, 11 ¥, 124". (2.64) 


Here, we have to take into account not only the product | Y, | | Y, | 
cf the amplitudes of the narrow-band signals, but also their phase 
cifference Ig, which determines the argument of the complex num- 
ber (2.64). The processes &(t), n(t) dealt with in radio engineering 
are real, and so are the transformations converting them into real 
ratrow-band processes Y,, Y,. This imposes the restrictions 


&A—) = 8), 8, (—4) = 87) (2.65) 
cng, and g,. If we set 


&(w) = £,(w) = gl |), 


where g is a real function, formula (2.61) gives 
CY (oo, ta) ¥olvo, to)> = Re S{E, nia] | gw) dwn. (2.66) 


We see that forming the product of two real, narrow-band 
processes cannot give complete information about the cross- 
spectral density as a complex function. To complete the specifica- 
tion of S[£, 7; wo], we need a second pair of narrow-band processes, 
but only one function in the first pair need be changed. Writing 


&(w) = all), 8, (w) = —e (lo |) sgne, 
we find from (2.61) that 
CY a te) Polos)? = Im SE, 95 a] [ g%w) do. (2.67) 


Thus, to experimentally determine the cross-spectral density, it Is 
not enough to form just one product of two narrow-band processes. 
Another product of the two processes must be formed, and before 
dcing so, we have to use a phase inverter to introduce an extra phase 
sh ‘ft (preferably of 90°) between the two processes. 


CHAPTER 3 


Gaussian and Non-Gaussian 
Random Processes. 
Quasi-Moment Functions 


{. Gaussian Processes and Their Probability Densities 


Suppose that all the correlation functions &,(t,, t,), R(t, ty, tg), --- 
»f a random process £(t) are zero, except the first-order correlation 
function k,(t,). Then, £(é) is not random at all, and is said to be 
deerministic, which means that all the realizations of &(#) are the 
same. Next, suppose that all the higher-order correlation functions 
of &(t) are zero, except the first-order and second-order correlation 
functions k,(t,) and &,(t;, t2). Then, the process £(z) is said to be 
Gdussian (or normal). In this case, according to formula (1.67), we 
have the following expression for the characteristic function of the 
n zandom variables €(t,), ..., &(t,): 


Oa. sts ts etn) = 089 FD faa) a ~ BDY ha) amyl Gt) 
Using (3.1), we can find the probability density 
Wr[E(ty), «0» E(tn)] 
by calculating the inverse transformation 
teal Gay oy bn) (3.2) 


" y 7@7 © pe 

= oa | wf exp |—i 2) tats Op (Up, ove) Uns byy voy ty) Gt. Gy , 
e 00 —2o 
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where €, = &t,); this is just the multidimensional generalization 
of the integral (1.19). The expression 2, , ke(t,, ts) u,#,, Which can 
be written as 


(1 [E(ta) — Ar(ta)] u.{ 


is a positive definite quadratic form. Therefore, the absolute value 
of the characteristic function appearing in (3.2) certainly does not 
increase as w, ..., u, get larger. This implies that regardless of the 
presence of the infinite limits of integration, the integral (3.2) 
exists, at least in the same sense as the integral (2.30). Actually, we 
can carry out the integration in (3.2) explicitly, obtaining 


Wy (Ey, oe) En) = (27)-*/? (det || Ro(ta, ts) I[]-2? 


n (3.3) 
x exp }— 22) daglfa — ttl] Es — AGI} 


where det || &,(t,, ts) || is the determinant of the correlation matrix 


Roty,t1) Ro(ty, ta) +.  Ro(ta, tn) 


I] Ro(ta, ta) || = c ; (3.4) 
Roltn, t,) Rilta; ts) ere Rota, ta) 
and || a,, || is the inverse of the correlation matrix, Le., 
II Gap I] = I] Rater ta) II-?, DY) aaphalta, ty) = Say (3.5) 
pel 


(5,, = | fora = y, 6,, = Oif a A y). A probability density of the 
form (3.3) is said to be Gaussian (or normal), and the random variables 
£,, ...) §, are said to have a Gaussian (or normal) distribution. 

If £(z) is stationary, and if we set n = |, the matrix (3.4) and its 
inverse each reduce to a single element, and in fact 


! | 
4 Rea) 
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[cf. (2.5)]. Therefore, the one-dimensional probability density 
equals 


ulé(e)] = = ep | — sox (ee) ~ mil. (3.6) 


Setting x = 2, we find that the inverse of the correlation matrix is 


| k(O) =. —k(r) 
7 ne k(r) [" _ | FO) —P@) BO) — Fo) G.7 
ee kr) RKO) ~j} —ke) ok) rs 
i RQ) — A*(7) RO) — RX(z) 
sot that the two-dimensional probability density is 
Lyte ft —2R(r) £2 + 
w(E,, fo) = AVIS? | ot OC RMT) (3.8) 


In certain problems where it is difficult to calculate integrals 
inyolving the probability density (3.8), it is convenient to expand 
(3, 8) in a series of powers of the correlation coefficient R(7). To 
finld such an expansion, we set n = 2, k; = m = 0 in formulas 
(3. 2) and (3.1), obtaining 


wi &) = gal ie exp — tu, — ish, (3.9) 
— _ {uy + 2R(7) Uj Uy + us] du, dus F 


Using the expansion 
! 


e-o Ruy, a yo = ” (o®Ruytty)? ’ (3.10) 


i p=) 


we find that 


2,,2 
—im,€, oa = 41 | du,| 


w(,, &) = = diy sf “ee 5 


2 
et see oe | du} 


_ x se S ue exp 
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Making the change of variables 
A, = ot, , Ag = Oto, 


and using the integral representation 
(p+) a Go? ae is i -+| 
F°?+)(z) = | _drexp | ide — Sf a (3.12) 
for the derivatives 


Fe+g) — ——_+ — —_ ee e722 (3.13) 


of the probability integral 


Lop 
Fay= ti’ eed 
@) ox] ms 


we obtain the expansion? 


w(fy, £2) = 4d ren (4) Fo+n (£2) a (3.14) 


Cg 


Formula (3.14) can also be written in terms of Hermite polynomials. 

Similar expansions exist for the higher-order normal distribu- 
tions, i.e., for the normal! distributions of order three, four, etc. 
For example, making series expansions like (3.10) of the exponen- 
tials 


E-FR jattyttg ’ e-O°R egtatts , eR gteyttg ; 
where 


Ri = Rt — t), Ry = R(t, — ts), Ris = R(t, — ts) 


See H. Cramér, Methods of Mathematical Statistics, Princeton University 
Press, Princeton, N.J. (1946), p. 290, where the result in question is attri- 
buted to early papers by Stieltjes and Sheppard. 
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we can write the three-dimensiona! characteristic function in the 
form 
a 2 al > o*P 
@3(u, Uo, Uy) = Xp | > (uy + uy + u5)| > al rm (Rygttytt,)? 
p=0 
(3.15) 
x >, (—I + (Rasta) y (1 rl — (Rygttyts)’ 
} 

{n calculating the Fourier transform of (3.15), each term with 


fixed values of p, g, r reduces to a product of three integrals of the 
type (3.12). Therefore, we find that 


“(Eis bay £4) (3.16) 
= or Fetran (— 1 | eee (=2 Ss Flora) (= 2) ae Baar 


| 

According to what has been said, the most important (and the 
myst time-consuming) step in finding a multidimensional probability 
density for a normal process consists in finding the matrix || dog || 
which is the inverse of the correlation matrix || k,(t,, tg) ||. The 
problem of calculating the inverse matrix is equivalent to the 

problem of solving the system of equations 


Ro(ty, 24) ¥1 + Re(tys te) Ve + -- + Relty tn) Mn = M1» 
(3.17) 


Roftny t1) V1 + Reltn, te) Ye + ++ + Re(tns tn) Yn = in > 


If we solve (3.17) with respect to 4, ..., ¥n, We obtain expressions of 
the form 


n 


’ va = > BapXp (a = J, oeey n) P (3.18) 
’ pel 


Using (3.18), we can easily find the quadratic form 


Dd, asl fa — Fa(ta)] (5 — Palte)] 


a,f=l 


eT Se 
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appearing in (3.3). In fact, we need only form the sum 


n 


2pfx] =D) tara = D aaXpaXe , (3.19) 
a,p=1 


a= 


and then set x, = €, — h,(t,). 

This method is applicable for any number of instants of time 
ty, ..-) fn, and remains meaningful! even when the points 4), ..., t, He 
arbitrarily close together. In this case, the finite probability density 
w,[E(ty), ..-» E(tn)] goes over into a probability functional. 


2. Conditional Correlation Functions of Gaussian Processes 


Suppose we know the values &(7)), ..., (7) of a random process 
&(t) at certain time instants T,, ..., T,. This information changes the 
initial data concerning the random process, in particular, its 
correlation functions and probability densities. The new, modified 
correlation functions and probability densities are said to be 
conditional (or a posteriori), as opposed to the original quantities, 
which are said to be unconditional {or a priori). Qualitatively 
speaking, as a result of knowing certain values of a random process, 
we obtain a new “a posteriori random process,” which is chatacter- 
ized by different statistical properties. It should be noted that if the 
a priori random process is normal, then so is the a posteriori random 
process, 

According to formula (1.39), the conditional probability density 
of the random variables €&(¢,), ..., €(t,), given the values 
£(T,), »», &(T,), is equal to 


wl E(t), oy Ete) | E(T1), »» (TI 
(3.20) 


= x Wr sf £(tr), eed &(t,), &(T)), oney £(T,)] ’ 


where 1/N is a normalization constant, which does not depend on 
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E(2,), ..., &(t,). The Fourier transform of (3.20) with respect to the 
variables &(1,),..., &(t,) gives the conditional charactertstic function 


Ovona(lay +++ Uy) = ~ | x J exp Ji matteo 


x + of (tr), 005 E(t,), &(T,), oeey &(T,,)] d&(ty) ane dé(t,). 


Expressing the last integral in terms of the a priori characteristic 
function @,, ,(t, ..., Up, Vy, »-, U,), Which is the (r + s)-dimensional 
Feurier transform of the joint probability density 


Wry EC), toey E(t,), &(T), aery &(T,)] ’ 


we find that 


J < 
Feral» vay Uy) = vor! ese | exp iy, vpt(T)} 
(3.21) 
. K Ong (yy voy Uy, Uz, ove) Ue) AV, ws Ay 


The factor 1/N in (3.21) is determined by the condition 
Dena (0, eery 0) => lL. 

The calculation of the integral (3.21) is completely analogous to 
the calculation of the integral (3.2). The only difference is that there 
is now extra dependence on the numbers 4, ...,4,, which can be 
regarded as patameters. According to (3.1), the characteristic 
function appearing in (3.21) can be written in the form 


O,4 (Uy, +) Up) Vzy oe, Vs) = EXP }i >) Falta) u, + i> ky (Ts) Up 
a=] &=1 


=D) alte, tp) watt + PD) Dd) koltar Te) up — $D, kl Tw Tp) vars} 
a, fm awl f=l a,pml 


(3.22) 


If we let || a., || denote the matrix which is the inverse of the 
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correlation matrix || k,(7,, T,) ||, then it follows from (3.21), by 
analogy with (3.3), that 


; r tr 
Oona (ty 0) Uy) = We oP i> k,(ta) ts. — +> Rotor tp) Uatts 
aml a,f=l 


—4 > igp| &(T) ~ k,(T.) — ‘x kT, ty) u,| (3.23) 


a, B= 
x [&7,) — kT) — >) R(T,, ta) us| , 


The factor 
(27)-"/? (det || kT, Ts) (lJ? 


has been incorporated into the factor 1/N’, which is independent of 
By cay a 

We now introduce the conditional correlation functions k,(t) and 
R(t, t’), in terms of which the conditional characteristic function 
(3.23) can be written in the same form as (3.1), i.e., 


Buona (Uy, +) Up) = EXP 


> A(t.) uy —+ > Re(ta tg) Ugg}. (3.24) 
oo a,g=l1 


Separately equating the coefficients of the linear and quadratic forms 
in (3.23) and (3.24), we obtain explicit expressions for k,(t) and 


kot, t’): 
R(t) = R(t) + 2, lt Tz) Gapl&(T's) — Ra(T)) 5 
a (3.25) 
Raft, t') = y(t, ') — Dy halt Ta) dapho(T pt’) 
a, pul 


These expressions completely determine the a posteriori random 
P pletely P 
process, just as in the case of any other Gaussian process.? 


2 Formula (3.25) can also be applied (with an obvious modification) to the 
case where we know a continuous realization of the random process on an 
entire interval, 
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‘As an example, consider a stationary process with k(t) = m, 
k(t, t’) = o R(t — t’), and suppose we know the value of the 
process at some time ¢). Then, according to (3.25), we have the 
following conditional mean value and conditional second-order 
certelation function: 


f(t) = m + R(t — to) {&(to) — ml, 


: (3.26) 
R(t, t) = o*[R(t — t’) — R(t — 4) R(t —1’)). 


Setting ¢ = ¢’ in the last formula, we obtain the conditional 
‘verlance 

- [DE(t)lcona = of! — R4E— fy). (3.27) 
Note that the a posteriori process is nonstationary. As t — t, +0, 
the statistical scatter of the values of £(t) becomes progressively 
smaller, since R(t — t,)—» 1 as t — tg-> 0. As |t — tg |», the 
correlation coefficient R(t — to)» 0, and any difference between 
the a priori and a posteriori processes disappears. 


‘3. Non-Gaussian Processes and Quasi-Moment Functions 


When the higher-order correlation functions k(t;, ta, t3), 
'Ri{ty, ta, ty, 24), ... are different from zero, we have a non-Gaussian 
(or non-norinal) random process. In this case, the expression (3.1) is 
,no longer sufficient, and in (1.67) we must also take account of 
-ternis for which s > 2. Although the characteristic function of a 
.non-Gaussian process can immediately be written as an expansion 
of the form (1.67), the calculation of the corresponding probability 
densities is difficult, since in general, the Fourier transform of 
.(1.67) cannot be calculated directly. 

‘In order to simplify the problem of finding the multidimensional 
distributions of a non-Gaussian process, we introduce the so-called 
quasi-moment functions® 


ba(ti, tay ty) » Da{tys ter tay tq)y «+ (3.28) 


*P, I. Kuznetsov, R. L. Stratonovich and V. I. Tikhonov, Quast-moment 
functions in the theory of random processes, Dokl. Akad. Nauk SSSR, 94, 615 
(1954); Theory of Prob. and Its Appl., English edition, 5, 80 (1960). 
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These functions occupy an intermediate position (so to speak) 
between the correlation functions and the moment functions, and, 
together with A,(z,) and &,(t,, t.), they also completely characterize 
the process. By definition, the quasi-moment functions are con- 
nected with the correlation functions by the relation 


i= 8) 


2, 


smd 


exp 


is < 
“st > RX t,, ooey t.) 2a rr) 24| 
Kyo Wal 


(3.29) 


a 3 nr 
=!+ py sl, > b(ta» eee tw) Barer By» 


Povey iam 


which holds for any values of 7, ty, ..., tny By) ++) Z_- Using (3.29) and 
(1.67), we can express the characteristic function in terms of 
quasi-moment functions: 


’ n 22 n 
On (Uyy -00) Uns by, «> Ey) = EXP i > ky(tz) Ua + > > ko(tq, tg) uaip| 
a=l a, p=l 
(3.30) 


x } +2 Dole sis Egy) see ars 


Formula (3.29) allows us to find explicit formulas relating the 
correlation functions and the quasi-moment functions. In fact, 
these formulas are the same as (1.69) if we set k,(t)=0O and 
k,(t,, t.) = 0. The lower-order quasi-moment functions 3, 44, b, 
coincide with the corresponding correlation functions. The 
difference between the correlation functions and the quasi-moment 
functions begins with 3,: 


bs = ky, b=, b, =k,, 
b, = ke t+ 10{kgk3}, . (3.31) 


Here, the symbol {...}, denotes the symmetrizing operation, just as 


in (1.69). 
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oe 
- To calculate the n-dimensional probability densities, we have to 
take the Fourier transform of the characteristic function (3.30): 


ten(E, aay Gay titer ta) 


= (2n)-" 4 i 


n 2 
i> Ry (ta) Ua + =r > k(t, tg) uatta| du, ... du, . 
aol *a, Bul 


Ey bud {1 a. > Pte on ty) Uy « 


a=! 


(3.32) 


* exp 


Interchanging the operations of summation with respect to s and 
integration, we can write the last integral in the form 


Wa(Err wry bi bay wees be) (3.33) 


* i Ke > 7, be tS (— =) eo (— x) WE vans En) 5 


where w® is the probability density of a Gaussian process with the 
same functions k,(t), &,(t, t’) as the original non-Gaussian process. 
According to formula (3.3), we have 


103(E yy ons En) = (2m)-1? [det || Fa(ta» tp) ||] exp 


—4 > aapkatp| , 


a, pul 
(3.34) 


where 
ty = €,— Ry(ta)- 


We can also write the probability density (3.33) in terms of 
generalized Hermite polynomials. Suppose we are given a quadratic 
form 


ox] = F > ap aX p (3.35) 


in several variables x,, X9, ..., X,. Then, with the coefficient matrix 
|| 2, || we can associate the generalized Hermite polynomials 


Hox}, Halt), Healt, (3.36) 
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defined by the formula 
Haol#] = evt*) ( = =) ( = x) e-ele] , (3.37) 
If we write 
AEA. 3 
= Yalx] = -2 2,p4p ’ (3. 8) 
then, according to (3.37), we have 
H, [+] = Yas 
H [x] = ya¥ 8 — aps 
H,,|x] = VaV Vy — FapVy — FayVp — 2pyVa» (3.39) 


A ypyl*] = VaV BV V8 — VabVyV5 — Fay AV5 — @adV Vy — Faas 
— 2p aVy — 2yoVaVp — Gap,s — Gay2p3 — 2a5Gpy » 


In terms of the generalized Hermite polynomials, we can write the 
probability density (3.33) in the form 


WiC ged Cait eti ta) (3.40) 


-)4+ 347 a. btay os te) He, al BCE) — Ru(ON| Ea ons En) 


Thus, we see that from a knowledge of the quasi-moment func- 
tions, we can immediately write down the probability distributions 
of any order as an expansion involving generalized Hermite 
polynomials. The greatest difficulty encountered in doing this is 
finding the matrix || 4,,|| which is the inverse of the correlation 
matrix || A(t, t,) |l- Only the first few terms of the expansion (3.40) 
play an important role, and if we restrict ourselves to a certain 
number of terms of low order, we can neglect the terms of higher 
order. 

In the special case where » = 1, we have a one-dimensional 


SEC. 3] GAUSSIAN AND NON-GAUSSIAN RANDOM PROCESSES 51 
t 

probability density. When the indices coincide, the polynomials 

(3.39) become ordinary Hermite polynomials, i.e., 


H,_ [x] = an lH (Ai, [2y ) = a™ 2H (ail) , (3.41) 
m times 


where 


H,,(z) = e#/2 (- ay ents, 


If we bear in mind that 


is Sh ne 
a, R(t, t) a? 


‘nthe one-dimensional case, then, according to (3.40) and (3.41), 
we find that the probability density is 


mel = fi + Bop ge a OSA] meen), 42) 
where 


Dg = Boke tf) w(£) 


t 


= ew |- 5 sax le — Aor]. 


The expansion (3.42) is known as Edgeworth’s series. ; 

, Next, we consider the two-dimensional case (n = 2). Then, when 
thé process is stationary, the matrix || a@,, || is given by (3.7), and 
tha probability density w} has the form (3.8). In this case, the 
variables y, and y, appearing in (3.39) are 


1 x, — Rx 1 —Rx, + »% 
Wale hap) 


a 


In the two-dimensional case, the indices in (3.39) can only take the 
values 1 and 2. If we write 


Ay 10. _ Hum) ’ (3.44) 


H times ‘mn times 
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then it follows from (3.39) that 
Ay, = Hy) = yi — @); 
Ay, = Ayyy = Va — Ae; 
Ag = Hoey = 9 — aap, 
Ay, = Hoo) = nH — 34,91 , (3.45) 
: Aye = Hey = vi¥e — 24291 — Ayes 
Ayo, = Huey = Wi — 2a,e¥_ — 49291» 
Hao = Hog) = ¥2 — 34z2y2 - 
We can write (3.45) as 
Hix) = = (ai - 1), Hay = sn +R), Hoy = = (23 - 1), 
out out out 
Hig) = — (73 — %), A) = ae (2325 + 2Rz, — 22), (3.46) 
oyu3 od 
1 1 
Age) = os (212% + 2R2_ — 2%), Hogg) = op? (23 — &), 
if we introduce the notation 


_ R? — ,2 
on ie (3.47) 
x, — Rx, ’ Ba — Gpyg = Ry +s PS 


oc _—— 
1 ORY on am 


Combining terms in the expansion (3.40) which differ only by 
permutations of the indices 1 and 2, we can write the two-dimen- 
sional probability density as 


wEn be) = |L +S Bam Hum (ECE) — AON] wa) 
yee (3.48) 
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where 
t Ilmes m llmes 
ne 


See 
Bin) = Bia m(tr, ooey th» to, oeey to) ° 


:An expansion of this type is essentially an expansion in orthogonal 
‘polynomials. 

: As we have just seen, the quasi-moment functions are the 
coefficients in an expansion of the probability density in an infinite 
series of generalized Hermite polynomials. This is true for any 
number of random variables £(t,), ..., €(¢,) corresponding to values 
of the original process. If the points 4, ..., t, are made arbitrarily 
‘close together, we obtain the corresponding expression for the 
}probability functional. 


CHAPTER 4 


Markov Processes 
and Related Processes 


‘1. Definition of a Markov Process. The Stochastic Equation 


Markov processes, or processes without aftereffect, are a 
convenient abstraction. Although the actual processes encountered 
in radio engineering are not exactly Markovian, it is sometimes a 
good approximation to regard them as being Markovian. This makes 
it possible to obtain many concrete results by using the effective 
mathematical methods of Markov process theory. We begin by con- 
sidering ideal point processes without aftereffect. Then, we look into 
the question of when, and in what sense, an actual process encoun- 
tered in radio engineering can be regarded as a Markov process. 
‘ Let x(t) be a random process, and let x(t,), ..., x(t,) be a set of 
its values at the consecutive time instants t, > t, > ... > th. 
Consider the conditional probability density of the value of x(t) at 
the most recent time ¢,: 

Wy [X(ty), --., X(tn)] 
w[x(t) | (ty), -.-, x(t,.)) = PS ATID ; (4.1) 
The process x(t) is said to be a Markov process (or a process without 
aftereffect) if the conditional probability density (4.1) depends only 
on the last value x(t,) and not on the preceding values x(t), ..., x(ty) 
[ta > tg, --+ te > tn]. Of course, (4.1) can still depend on -(%,), 
t,, tg, and hence for a Markov process, we can write 


WH, | Xp +01 Xe) = Peyty(%r%2) ("> 2), (4.2) 
“where 8S AE at he = ee) 


55 
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If we bear in mind that by definition, the conditional probabilities 
(4.1) corresponding to different values of n are always connected by 
the compatibility relations 


| oe | Way cory Xn) W(x, oes Xn) dx, = w(x; | Vay corr %n—1) (Xo, seep Xn_1)s 
(4.3) 


then it can be seen by substituting (4.2) into (4.3) that the same 
function p, ; (x,, *2) appears in (4.2) for all x. This function equals 
the conditional probability density 


Peyu(%p %2) = W(% | %2) (ty > 42), (4.4) 


and is called the transition probability. The definition of conditional 
probability implies the formula 


W(X, v00) Xq) == W(X, | Hyp --+y Xp) W( Hy, | Hg, --ey Hp) --e W(X p_y | Xp) W(%Xp), 


(4.5) 


from which it follows by using (4.2) and (4.4) that in the case of 
Markov processes, the multidimensional probability densities factor 
into products of transition probabilities, ie., 


WU Xy, -20y Xn) = Ppig(Hr Xe) Ptgtg(%er %g) «++ Pt, st (%n—1» Xn) W(%n)-- (4.6) 


Thus, if we know the univariate probability distribution and 
transition probability of x(t), we can write any multivariate prob- 
ability distribution of x(t), i.e., the functions p,, (x, x") and w[x(t)] 
completely characterize a process without aftereffect. 

The transition probability has to satisfy certain conditions. In the 
first place, there is the normalization condition 


| Puls, x) dx = 1, (4.7) 


which follows from (4.4), say. Moreover, integrating the distri- 
bution (4.6) with respect to some intermediate value x(t,), 
1 <k <n, we obtain a probability density w,_, of lower order, 
which in turn satisfies an equation like (4.6). In fact, we have the 


set. 1] MARKOV PROCESSES AND RELATED PROCESSES 57 


tollowing integral equation, called the Smoluchowski equation (or 
thé Chapman-Kolmogorov equation): 
| Peele 4) Ptyta(%e» Xg) dx, = Put lev a) (t) >t, >t). (4.8) 


Yo obtain (4.8), we need only consider the relation 


J eal(es)s (62), 2(¢5)] de(te) = wal(ty), #(t0)] (49) 


and then write w, and w, in the form (4.6). 

Next, we study the time dependence of the one-dimensional 
probability density. Setting » = 2 in (4.6) and integrating with 
respect to x,, we obtain the equation 


to(%1» th) = | Pegs *rs 2) 0% fa) dae, (4.10) 


where 
w(x, t,) = w[x(t,)], t) > tg. 


To convert (4.10) into a differential equation, we choose t, close to 
t,, Setting 


t=, tf, =t+4+7, vy =x, xy =x, 
we can write (4.10) in the form 
t02(%) = | Pes eer x) (2) dx. (4.11) 
- We now introduce the characteristic function 
Olu; x) = Cetuterary = [ etultm prs, x)dx, (4.12) 
of the random increment x, — x which occurs during the time 


interval [t,¢-++ 7], given that x(t) = x. Substituting the inverse 
transform 


1 . 
Ptart%n *) = sf e~iulzs—2) O(u; x) du (4.13) 
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into (4.11), we obtain 
ay ee J J e-M2,-2) Ou; x) du w(x) dx (4.14) 
wets Qa ’ ®. 2 


According to the usual formula (1.22), the characteristic function 


(4.12) equals 


@(u; x) = 1 > ey m,(x) (4.15) 


in terms of the moments 
m (x) = (x, — x)*) (4.16) 


of the increment x, — x. It follows that 


w(x,) = > oa 5 if] e~*u(2,—=) (a10)* du m,(x) w(x) dx. (4.17) 


However, since 


5. | e—tulz,—z) (ru)$ du= (-— ae) + | e~tulz,—-2) dy 
(4.18) 


=(-Z) 9, 


we find that 
to_() = w(x) + y F(- ZY ime weg). — 4.19) 


Dividing by 7 and passing to the limit + —> 0, we obtain 
— | Qe \° 
He) = + (— x) [K@) wa), (4.20) 


s=] 


where the overdot denotes differentiation with respect to ¢, and 


K,(a) = lim ©), (4.21) 


sec. 1] MARKOV PROCESSES AND RELATED PROCESSES 59 


provided that the limits (4.21) exist.! Equation (4.20) is called the 
stochastic (or kinetic) equation. In the case where the sum of deriva- 
‘ives in the right-hand side of (4.20) has an infinite number of terms, 
the sum is equivalent to an integral operator, but otherwise (4.20) 
is a partial differential equation. 

According to (4.21), the moments m,(x) = <(x, — x)*) depend 
on 7 in the following way: 


m,(x) = K,(x) 7 -+ O(7*), 
m,(x) = K,(x)r + O(7), (4.22) 
mx) = K,(x) + + O(7 ) ’ 


Using the formulas (1.24), we can easily obtain completely analo- 
gous relations for the cumulants k,(x) of the random increment 
x, — x: 


kx) = K(x)r+0(*) (6s = 1,2)...). (4.23) 
If we define é(t’; x) as the derivative 


E(t's x) = E(t’) = a (t’ > 1) (4.24) 


of the conditional random process x(t’) which takes the value 
xit) = x, then 
° t4r 
wae J £(t') dt’ , (4.25) 
t 


4 
and the cumulants k,() can be expressed in terms of the correlation 
functions k,(t;, ..., t,)¢ of the process £(t): 


t t4T (+t 
, Ay = J ae J Ri(tyy oy tae dt, dt, (8 = 1,2,.). (4.26) 
t t 


1It should be noted that (4.19) and (4.20) have a simple operator inter- 
pretation. For example, (4.19) can be written in the form 


7) 
w, = e(i—,x)w, 


i; we agree that the second operator x in the function 6 of the operators 
i(@/@x) and x always acts before the operator i(8/2x). 
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Comparing (4.26) and (4.23), we find that the correlation 
functions &,(t,, ..., t,)¢ have delta-functions singularities of the form 


Ry(ty, oo tee = K,(x) 8(ty — ty). (ty —t.) +A, (8 = 2,3, +4), 
(4.27) 


where the A, denote other possible functions which are less 
singular than the first term (with spectra that fall off faster as the 
frequency is increased). If we substitute (4.27) into (4.26) and 
integrate, the functions A, give rise to terms of order r?, r°, ... 
The functions K,(x) are called the intensity coefficients of the 
derivative (4.24), which correspond to the given value x(t) = x at 
the given time t. These intensity coefficients appear in the sto- 
chastic equation (4.20), and in nontrivial cases, at least one of 
them is different from zero. As the above considerations show, 
Markov processes are intimately related to certain special random 
processes whose correlation functions contain delta functions. 
Such random processes will be said to be delta-correlated. 


2. Delta-Correlated Processes 


Suppose we have a delta-correlated random process £(t), described 
by the correlation functions 


R,(ty, ..-) ts) = K,(t,) 5(t, — ty)... 8(t, — t,) (s = 2,3,...), (4.28) 


where the K,(t) are the intensity coefficients, which in general 
depend on the time. In the special case where the process 
&(t) is stationary, the coefficients K, are constant. Setting 
t, = t, =... = t, in (4.28) we see at once that a delta-correlated 
process has infinite cumulants, This shows that delta-correlated 
processes are not encountered in radio engineering practice. 
Nevertheless, such processes are sometimes useful as approximate 
or auxiliary working concepts. 

A special role is played by stationary Gaussian delta-correlated 
processes. Let &(t) be such a process, with zero mean value. Then, 
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among the functions (4.28), only the second-order correlation 
function 


Ro(t, — te) = K,&(t, — te) (4.29) 


iS nonzero, ic., ky = K; = Ky=... = 0, and according to 
(4.29) and (2.13), €(t) has the constant spectral density 


: Slé;0] =2K, «w)=K,=K. (4.30) 
‘Therefore, é(t) will sometimes be called Gaussian white noise, or 
simply white noise, by analogy with white light, which has a 
constant spectral density. 

The spectral density of any process encountered in practice is 
only constant over a certain range, and there is always some cutoff 
frequency w, which is an upper bound for the frequencies appearing 
in the spectral density of the process. This means that the for- 
mula [cf. (2.15)] 


Dé =f vais (4.31) 


does not give an infinite value for the variance of the process, but 
rather a finite value, whose order of magnitude is Kw,/7 ~ K/r coy. 
Replacing the actual process by a delta-correlated process means 
that this frequency w, is not explicitly taken into account. This is 
permissible if w, is considerably larger than all other frequencies 
which are important for the given system or problem. On the 
other hand, the correlation time 7,,, = I/w, will then be small, 
in fact, smaller than all other relevant time constants of the system. 
Under these conditions, we can make the substitution 


halle — 41) > 84a — th) [als dr (4.32) 


where the coefficient of the delta function is chosen in such a way 
that both sides of (4.32) give the same result when integrated with 
respect tot, — ¢. 
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3. The Fokker-Planck Equation and the Kolmogorov Equation 


A Markov process is said to be continuous if its higher-order 
intensity coefficients K;, Ky, ... equal zero. In this case, equation 


(4.20) takes the form 
, Q RE 
(x) = — (KG) w(x) +555 (Ke) o@)], (4.33) 


and is called the Fokker-Planck equation (or the diffusion equation) .® 
[Introducing the probability current 


G(x) = Ky(x) (x) — 5 5 [Ks) (9), (4.34) 


we can write the Fokker-Planck equation in the form 
. , OG 
+s =0, (4.35) 


which can be interpreted as the equation of conservation of prob- 
ability. The probability current (4.34) describes the amount of 
probability crossing the abscissa x in the positive direction per 
unit time. Consider the interval x, << x <x,, with end points 
; and x,. Then G(x,)r is the amount of probability entering this 
interval in time 7 across the abscissa x,, and G(x,)r is the amount of 
probability leaving the interval in time 7 across the abscissa x». 
Jf probability never disappears inside the interval x, <x < %,, 
end if there are no sources of probability inside x, < * < x,, then 


2 Concerning the physical meaning of the Fokker-Planck equation, see e.g., 
I.. S. Pontryagin, A. A. Andronov and A. A. Vitt, On the statistical analysis 
of dynamical systems, Zh. Eksper. ‘Teor. Fiz., 3, 165 (1933), reprinted in 
A. A. Andronov, Selected Works, Izd. Akad. Nauk SSSR, Moscow (1956), 
p. 142; H. A. Kramers, Brownian motion in a field of force and the diffusion 
ntodel of chemical reactions, Physica, 7, 284 (1940); P. I. Kuznetsov, 
F.. L. Stratonovich and V. I. Tikhonov, Correlation functions in the theory 
of Brownian motion. Generalization of the Fokker-Planck equation, Zh. 
I'ksper. Teor. Fiz., 26, 189 (1954). 
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‘the difference G(x,)r — G(x,)r is the increment of the total 
' probability Se, w(x) dx in the interval x, < x < x, Le., 


| in w,(x) dx — f. w(x) dx = G(x.) 7 — G(x) 7. 
? cia 
1 Dividing this equation by 7 and x, — x,, and passing to the limits 
it + 0, x. — x,» 0, we obtain the equation of conservation (4.35). 
: Thus, just as the equation of diffusion or heat conduction involves 
a flow of mass or heat, the equation (4.35) involves a flow of 
.probability. In fact, from a mathematical point of view, all these 
, phenomena are described by an equation like (4.35) or (4.33). 
In every particular realization of a Markov process, the trajectory 
‘x(t) has a very complicated form, and can be thought of as being 
‘swept out by a ‘“‘representative point,’’ which moves along the 
x-axis just like a Brownian particle or a particle undergoing 
diffusion. If we take a large number of realizations of the random 
“process, we obtain a large number of ‘“‘representative points,” which 
‘move about in a random and erratic fashion. These points form a 
kind of ‘‘gas’’ which undergoes diffusion, and whose density 
at any point is proportional to the probability density. Each 
separate point represents a ‘‘molecule’’ of the “‘gas,” and as they 
move about, ‘‘molecules’’ cannot be created or destroyed. Thus, 
in this model, we can talk about the ‘number of moving points” 
instead of the ‘‘amount of probability,’ and we can treat Ps w(x) dx 
‘as the number of points in the interval x, <x < x,, or as the 
amount of time that a single moving point spends in this interval. 
Of course, in this language, we have to talk about the relative 
number of points in an interval (rather than the total number), or 
the relative amount of time spent in an interval by a single point, 
since these are the quantities corresponding to probabilities. 

In order to obtain solutions of the Fokker-Planck equation (4.33), 
we have to supplement it with initial conditions and boundary 
conditions. If we specify an arbitrary initial distribution 


w(%, to) = w(x) 


at some initial time tj, we can find the subsequent evolution of this 
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distribution [i.e., the function w(x, t) for t > t,] by solving equatior. 
(4.33). If the initial distribution is a delta function, so that 
W(x) = 5(x — x9), then the resulting probability density is just 
the transition probability p,, (x, x»). Therefore, the transition prob- 
ability can be found as the solution of the equation 


a = —— a [K1(*) Per(* %o)] +3 2 a 2 5 (Kx) Pu{*, ¥0)] » (4.36) 


with the initial condition 
: Piota(®s %o) = B(x — Xp) - (4.37) 


It can be shown’ that regarded as a function of x9 and fo, the 
transition probability p4,(*, x») satisfies another differential 
equation 

OP ss,(%, Xo) OP iy,(%, Xo) 


Ot ees ax, Ky (x i(%, = 


OD 44,(x, Xo) 


Oxo 


Ky). (4.38) 


This equation, which is the adjoint of equation (4.36), is called the 
Kolmogorov equation. In the case of a stationary random process, 
K,(x) and K,(x) do not depend on ¢, and the transition probability 
Piatt = p, depends only on z and not on ¢. Then 

Opt, Pu, 


—— ent 


and the Kolmogorov equation (4.38) gives a second expression for 
the derivative 2p,/@r, L.e., 


Op. — opAx, ) 1 a *p (x, x) 
ee re K,(*.) + 5 on ee K,(x9) ; (4.39) 


in addition to the expression for ép,/ar given by (4.36). 


3See e.g. A. T. Bharucha-Reid, Elements of the Theory of Markov 
Processes and Their Applications, McGraw-Hill Book Co., Inc., New York 
(1960), Chap. 3. The Kolmogorov equation is often called the backward 
equation, and the Focker-Planck equation the forward equation, for an 
evident reason. 
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Next, we consider the boundary conditions which have to be 
taken into account when solving the Fokker-Planck equation 
(4.33) or (4.36). If the function x(t) can take all possible values 
f-om —o tO o, the Fokker-Planck equation is valid on the whole 
infinite x-axis, and then the boundary conditions take the form 
of conditions at +. Integrating (4.35) with respect to x from 
--co tO oo, and bearing in mind that the normalization condition 
j w(x) dx = 1 is satisfied identically for all t, we easily see that the 
condition 
G(—«, t) = G(o, t) (4.40) 
must hold. However, in addition to (4.40), the stronger conditions 


C6.) = 66):=0 (4.41) 
and 

w(—co, t) = w(oo, t) = 0 (4.42) 
are usually satisfied. According to (4.41), the ‘representative 
points” of the process cannot appear at infinity or leave at infinity. 
In cases where the function x(t) can only take bounded values 

lying in some interval 
4X SM, (4.43) 


we only consider the Fokker-Planck equation in this region, and the 
boundary conditions take the form 


G(x, 1) =0, G(x, t) = 0. (4.44) 


According to (4.44), there is no flow of “representative points”’ 
across the boundary, i.e., no random trajectory can enter the 
region (4.43) by crossing the boundary, and every random trajectory 
terminates when it arrives at the boundary. 

Of course, depending on the problem under consideration, 
we can have other boundary conditions as well. For example, 
i Vol. II, Chap. 9, we shall consider a case where the 
koundary condition takes the form of a periodicity condition 
ox + 2, t) = w(x, t). The question of boundary conditions is 
also approached differently in first-passage time problems, which 
will be discussed below. 
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4. Solutions of the Fokker-Planck Equation 


The probability density w(x, ¢) satisfying the Fokker-Planck 
equation (4,33) is uniquely determined by the prescribed initial 
and boundary conditions. If the coefficients K,(x) and K,(x) do 
not depend on the time, then, as time increases, the distribution 
w(x, t) usually approaches a stationary distribution w,{x), which 
does not depend on the initial distribution w(x, t,) or on the time ¢: 

(x) = 0. 
Setting w,, = 0 in (4.35), we find that 
G,dx) = Gy = const, (4.45) 


i.e., the probability current is a constant, independent of both 
x and t, According to (4.34), for a fixed value of G,,, equation 
(4.45) is a linear differential equation in w,,: 


F [Kals) 9,42)] —2K,(x) ws) = —2Gy. (4.46) 


The general solution of (4.46) can be obtained by ordinary 
methods. If we write K,(x) w,{x) = v(x), then (4.46) becomes 


a _ 4 Ai 
Ox K, 


Multiplying (4.47) by 


0-2) ay 4 


and integrating with respect to x, we find that (4.47) has the 


solution 
ze K, 
2f K, dy|., 


v= —2Gy- (4.47) 


fd z K, : 
u(x) = —2Gy]| exp {2 J K dy| dx’ + Cexp 
xy xz’ 2 


from which it follows that 


os) = RPS ae Ol Re LPP Se ol 
(4.48) 
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\ 
Here, C is an arbitrary constant of integration, and the limit of 


integration x, is also arbitrary. However, apart from G,,, there is 
essentially only one arbitrary constant in the solution (4.48), since 
changing x, is equivalent to changing C. The choice of the constant 
C (or equivalently, of x,) is determined by the normalization 
gondition, while the probability current G,, is found from the 
boundary conditions. If we choose (4.41) or (4.44) as the boundary 
éonditions, then the expression (4.48) simplifies to 


a Ze * Kyy) 
W(x) = Ki@) exp }2 J. Kx(y) dy. (4.49) 


Thus, from a knowledge of the functions K,(x) and K.(x), we can 
immediately write down the stationary distribution in terms of a 
quadrature. This shows the effectiveness of stochastic methods 
based on the Fokker-Planck equation. Unfortunately, investigation 
of ‘transient processes,”’ which involves calculation of the transition 
probability by solving the nonstationary equation (4.33), is a 
much more difficult problem. However, in certain special cases, 
ithe problem can be simplified, e.g., when K,(x) is a constant and 
\K,(x) = ax + b is a linear function of x; then, the process is 
Gaussian and can be studied by using the theory of Gaussian 
processes. 

In some instances, the nonstationary Fokker-Planck equation can 
be solved by applying the method of separation of variables, 1.e., we 
look for a solution of the form 


w(x,t) = X(x) T(t). 
Then, dividing both sides of equation (4.33) by w(x, t), we obtain 


T(t) _ 


T(t) 2 


a 3. 
— £ 1K) X(@)] + 5 33 [Kale) X@)]| XG) = A, (4.50) 
where A is a constant. For certain prescribed boundary conditions 
(usually zero boundary conditions), the resulting equation 


; & [K.(x) X(x)] — 2 [K,(~) X(~)] +AX(%)=0 (4.51) 
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has solutions consisting of a sequence of eigenfunctions X,(x), 
X,(x), X,(x), .... with corresponding eigenvalues Ay, A,, Ad, -. 
Integrating (4.51) with respect to x, we easily obtain 


Am if X,(x) dx =0, (4.52) 
if the difference in the currents 
ex.j— —1 21K K,X 
[Xnl = —a5, | oXm] + 144m 


evaluated at the limits of integration (i.e., at the boundaries) 
vanishes. It follows from (4.52) that either A,, = O or [X,,(x) dx = 0. 
The value X = 0 is actually an eigenvalue, and in fact, for A = 0, 
equation (4.51) reduces to the stationary equation. Thus, the 
eigenfunction X, corresponding to Ay = 0 is just the stationary 
distribution (4.48) or (4.49) found above, i.e., 


Xo(x) = ws(x) - (4.53) 


The other eigenfunctions X,, satisfy the condition 
J X,(x) dx =0. (4.54) 


[n terms of the eigenfunctions, we can write the solution of equation 
(4.33) in the form 


ro(x, t) = TyXq(x) +) Te tat (x) (4.55) 


m=] 


where the coefficients 7,, are calculated by using the initial 
conditions. To calculate T,,,, it is convenient to use the orthogonality 
of the eigenfunctions (with weight I/w,,), as expressed by the 
formula 

dx 
Welt) 


J Xn) Xa(2) 3 mn (4.56) 


\uxe assumed that d,, 4 A, if m 4 2.) 
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* To establish the orthogonality relation (4.56), we write (4.51) 
for two different eigenfunctions: 


\ a a 
Jy ClXml = AmXms 52 G[Xa] = Xn 


where 

GX] = —5 2 (KX) + KX 
We multiply the first of these equations by X,/w,, and the second by 
Xi/w,, and then integrate with respect to x. The result is 


4 
[uz G[Xq] = An [ X%iXnS =, (4.57) 


a dx dx 
J Xm 5g Xn] = Oe J XnXn (4.58) 


Next, we prove that the left-hand sides of equations (4.57) and 
(4.58) are equal. Consider the integral 
Q l 3 Q 
1= f gx Glmufldx = —5 | Fg [Kemufl dx + |e [Kivull, 
which can be written in the form 


l 
| ; —F eK sj” oF dx —fex 2 [Kyou 3 M gx — te t& [K2%,,] dx 


rs) rs) 
+ [ eK. 2 ax 2 i [ea [Kyev,,] dx. 


The third and fifth terms vanish because of the fact that the 
stationary distribution w,; satisfies the equation 


i 

: 8G[w’,:] =i 
ox 

Moreover, it is clear from (4.46) that under the condition 


Glu] = 0, (4.59) 
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the second term equals 
7) 
=] | Kien g dx, 


and can be combined with the fourth term. Therefore, we have 


Integrating the last equation by parts, we obtain 


b= — 4 So kyalsde t+ [ Z tk woul fae = f LGtwuel fae, 


provided that 


Pel og 
8 ax — ax) = 9 
or 
gGlwf] — Gluself = 9 (4.60) 


on the boundary of the region under consideration. 
If we now set g = X,/w,,, f = X,,/wW,, we see that the relation 


[xox + fx, 2ex12 (4.61) 
holds, i.e., the left-hand sides of (4.57) and (4.58) are equal, as 
asserted. For this to be true, it is sufficient that the condition 


(4.59) and at least one of the boundary conditions 


G[X] =0 or * =0Q onthe boundary 


st 


be satisfied. It follows from (4.57), (4.58) and (4.61) that 
dx 
(An —< a) | XmXn se =0, 


which proves that the eigenfunctions corresponding to different 
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cigenvalues are orthogonal, with weight 1/w,,. Here, we assume 
that each eigenfunction is normalized in such a way that 


[wai (m = 0, 1,2, .). 
st 


Thus, we have proved the orthogonality relation (4.56) in the case 
where there exists a stationary distribution w,, satisfying (4.59), 
and no two eigenfunctions have the same eigenvalue. In most cases 
encountered in practice, the system of functions X,,(x) is 
complete. * 

The coefficients T,,, appearing in the expansion (4.55) can now be 
expressed in terms of the initial distribution w(x, t)). Setting 
t = ty in (4.55), multiplying both sides of the resulting equation 
by X,,/w.:, and integrating with respect to x, we obtain 


dx 
Pn = J w(x, ty) X,,(2) walx) (4.62) 
where we have used the orthogonality relation (4.56). If the initial 
distribution is a delta function, i.e.; if 


w(x, ty) = 8(% — x), 


then the distribution w(x,t) is just the transition probability 
Diz,(% Xo), and in this case, according to (4.55) and (4.62), we have 


Pu, #9) = Sy AaE) Kaleo) ryt (4.63) 


sBy using the transition probability (4.63) and the initial distri- 
‘bution, we can write probability densities of any order. When the 
initial distribution is stationary, the expression for the two- 
dimensional probability density w,(x, x) = Py y,(%, Xo)Wsf%o) is 
especially concise: 


t 


c 


W(x, %) = > Xx) Xoy(%p) ent! (4.64) 
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In particular, this implies that the correlation function and spectral 
density of the random process y(t) = F[x(t)] are given by 


k,(7) = > he e- Amal | (4.65) 
and 
spiel = >, (4.66) 
where 
= | F(x) Xp(x) dx . (4.67) 


Example 1. Consider the equation 


K dw 
=< (pew ae a 2 a2" 


In this case, (4.51) takes the form 


w 


aX oe 
ee + ROX) + 3X =0 (6 = 55): (4.68) 


If we impose zero boundary conditions at x = + o, then, according 
to formulas (7.355.3) and (7.351) of Ryshik and Gradstein’s hand- 
book,‘ equation (4.68) has eigenvalues A,/B = n (n = 0, 1, 2, ...) 
and eigenfunctions proportional to 


Fiat) (=) 


dt 
Vv = da™ 


Thus, if we make the appropriate choice of normalization constants, 


where 


eB /2 . 


Fin+)) (g) = 


X,(0) =a Fe (2), Ay = mB, (4.69) 
and (4.64) is identical with the expansion (3.14), where R = e7#t"!, 


“I. M. Ryshik and I. S. Gradstein, Tables of Series, Products, and Integrals, 
VEB Deutscher Verlag der Wissenschaften, Berlin (1957), pp. 405, 404, 246. 
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‘Next, we consider equation (4.68) for x > 0 (instead of 
+o <x <o), with the boundary condition GLX),_5 = 0. In this 
case, we can only keep the eigenfunctions (4.69) which have even 
indices, i.e., 


2 I amn+ —_ 
Xmn(X) = “~/Qmyr of (=) ’ Am = 2mB > (4.70) 


and the expansion (4.64) now becomes 


w,(x, Xo) = 


am. ast Fame) (=) Fom+n (2) ¢ e-tmBlel, (4.71) 


Example 2. Consider the equation 


0 Sle £)4 +4. 


which describes a Rayleigh process (see Chap. 7, Sec. 3). The 
corresponding equation for the eigenfunctions is 


) ox 29 (X) 2) K 
DBS + ge OX) — oF 5. | ge) t gx = 0 (o* = 35): 
j 
which becomes 
* ax aX | re +(A/2p) 
Sa tae + PR + al * =o. G72) 


after we make the change of variables z = x?/20?. If we impose zero 
boundary conditions at x = 0 and x = o, then, according to 
formulas (7.302) and (7.142.1) of the handbook cited above,® 
equation (4.72) has eigenvalues A, = 2nB (n = 0, 1,2,...) and 
eigenfunctions proportional to 


atie*L (2), 


where 


a” 
L,(2) =e 5 (ae) 


6]. M. Ryshik and I. S. Gradstein, op. cit., pp. 396, 381, 247. 
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are the Laguerre polynomials. Therefore, we have 
_ L 23/20? x 
Xx) = ert*L, (+5), (4.73) 


and the expansion (4.64) takes the form 


— 20 9-tat423) 20 5 ete, ee xo —2nAltl 
il a alr a al p» cris ln (Gor) En (Ger) € eT) 


Example 3. Consider the somewhat more general equation 


o2l-94+43. 


with zero boundary conditions at x = 0 and x = ~, The corre- 
sponding equation for X is 


ox ce) 2y 0X Mae 2, 
ora ta OX) — oe a (Z) tex =0, 
which takes the form 
@u du ete t28), $e 
fea tet EEO Seo, 475 


where 4 = (2y/K) — 1, after we make the change of variables 
x 


7 = 7gF 


Again using formulas (7.302) and (7.142.1) of Ryshik and Grad- 
stein’s tables, we find that equation (4.76) has eigenvalues A, = 2n8 
and eigenfunctions u,(z) proportional to 


uaz 2tXx, 


ghtt e*L@n)(z) . 


Returning to the old variables and normalizing the eigenfunctions, 
we obtain 

1 gAutl 
V nl Pn + 2e $1) T Qe +1) 22802 


x2 
X,(x) = eW8/Ro? [30 (5 
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Then, according to (4.64), we have the expansion 


’ ae 2 xity ett ~(2*+ 25) /20% . a 
mes, Ha) = 5 (3) © p>, al D(n + 2p +1) PQp +1) 


(4.77) 


«19 (eno (a) 


which is a generalization of the expansions (4.71) and (4.74). 


5. The Multidimensional Fokker-Pianck Equation 


The considerations given in Sec. 1 of this chapter can be 
immediately generalized to the case of a multidimensional Markov 
process, which consists of several random functions x,(t), ..., %(£). 
Such a process is described by a transition probability 


: Pu'lxy(t), ---, Xm(t); *(t’), -- X(t"), 


if terms of which we can write the multidimensional probability 
densities. This transition’ probability is just the conditional 
probability density 


(1, cory Mm x4, ees Xm) 


4 
W(X), +++, X,) 


(4.78) 


Digg yiietey He My hh hee) 


of the random variables x,(t) = *,, ..., x,(¢) = *,, at a general time 
t, given the fixed values x,(t’) = x}, ..., x(t’) = x, at a previous 
time t’ < t. In the case of a continuous Markov process, the one- 
dimensional probability density w[x,(t), .... x,(t)] satisfies the 
multidimensional Fokker- Planck equation 


7 oer, Xm) = 2 — [K.(*1, oory Xm) w(x, oor Xm)] 
(4.79) 


l< o8 
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where the intensity coefficients K, and K,, are defined by the 
formulas 
| 
K,= ate Sax — a , 
(4.80) 
ee 
Kap = ae (ar — Xa) (Xp. — x,)> ’ 


by analogy with (4.21). 
In a special case, which might be called the case of ‘‘isotropic 
fluctuations,” the matrix || K,, || has the form 


Ka, = K8ba,, (4.81) 


and then the Fokker-Pianck equation (4.79) can be written in the 
form 


(4.82) 


where 


Gale) = Kal) (2) — 5 a [K(x) w(x)] (4.83) 


are the components of a probability current vector G = (Gj, .... Gp) 
in m-dimensional space (x denotes the set of coordinates +), ..., x). 
As can be seen from (4.82), the stationary probability density w,, 
satisfies the equation 


> Dee i (4.84) 
a=] 


However, if m> 1, the probability current G does not have to 
vanish inside the region R under consideration, even if G satisfies 
zero boundary conditions 


G,(*x)=0 (a= 1,..., m) (4.85) 


on the boundary of R, since rotational probability flows can occur. 


sEc. 5] MARKOV PROCESSES AND RELATED PROCESSES 77 


In, fact, the current G vanishes in the whole region R, ie., 
| 


f K,(*) Wax) — ; a ~— [K(x) w,{x)] = 0 (4.86) 


only in a special case which we call the potential case. 
If we substitute 


K(x) w(x) = e- U™ (4.87) 
into (4,86), we find that 
ou Ky 
we eR (4.88) 


This shows that in the potential case, the quantities 


K(x) 
K(x) 


a = l,.,., m) 
are the components of the gradient of a certain function, and hence 
satisfy the conditions 

OY Ky. 8 4 
(=e = (4) (4.89) 
If the conditions (4.89) are met, the function U is given by the line 
integral 


U(x}, -. ‘%n) = Sle 


ve Kid t. + Ky bm) + C, (4.90) 


where a), ..., @,, and the additive constant C are arbitrary. In terms 
of the function U, the stationary probability density is given by the 
formula 


l 


Wo(X,, sass Xm) = Ky, Xm) e Um... =m) , (4.91) 


where the additive constant in (4.90) is determined from the 
normalization condition 


| = J Wa Xp vo0y Xp) 1%, 22 Big = 


78 GENERAL THEORY OF RANDOM PROCESSES [cu. 4 


The solution (4.90) and (4.91) is an obvious generalization of 
formula (4.49). If the “potential conditions” (4.89) [i.e., the 
conditions corresponding to the potential case] are satisfied, we 
can always write down a stationary distribution (4.91) such that 
the Fokker-Pianck equation is satisfied and the current (4.83) 
satisfies the zerc boundary conditions (4.85). If the boundary 
conditions are different, or if the potential conditions (4.89) are 
not met, the problem of finding the stationary distribution 
becomes much more complicated. 

If the coefficient K does not depend on xj, ...,.x,,, the conditions 
(4.89) reduce to the conditions 

0K, 0K, 


Ox, O%q’ )) 


which are to be expected if we regard the K, as components of a 
force which can be derived from a potential. 

* Next, we generalize the above considerations to the case where 
the matrix || K,,|| is arbitrary, rather than a multiple of the 
unit matrix, as in (4.81). The absence of probability current is now 
expressed by the conditions 


Ce _ 
Gz = Kity —5 p> an [Kags] =0. (4.93) 
Setting w,, = e~¥ in (4.93), we find that 


- OKg . 
>, Kare = > ae st OK (4.94) 


Solving these equations for the derivatives 2U/dx,, we obtain 


= p2 » Ay, me: p> AyaKa (4.95) 


where |] A,, || is the inverse of the matrix || K,, ||, 1.¢., 


> A, Kap aa 8,8 


ya Il 
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It, fdllows from (4.95) that the potential conditions now take the 
forra 


z) : OK, K. 
Fag te (Gt — 2K) = GE Aa (St — 2K.) 
(y,6 = I, ... m) . 


If these conditions are met, and if there is no flow. of probability 
through the boundary of the region R, then the assumption that the 
probability current vanishes everywhere in R is justified, and we can 
find the stationary probability density by calculating the potential 
function U from its gradient (4.95). To do so, it is necessary that 
the matrix || K,, || be nonsingular so that it has an inverse. 
Regardless of whether the potential conditions are met, the 
solution of the Fokker-Planck equation can also be simplified in the 
case where K,,...,.K,, are linear functions of the arguments 


xy) +++) %,, and the functions K,, are independent OP Wisi Met 


(4.96) 


6. First-Passage Time Problems 


The apparatus of Markov process theory enables us to solve 
ima@ny problems involving processes which eventually terminate; 
this category includes first-passage time problems, which we shail 
cxamine only in the one-dimensional case, where the region under 
consideration is an interval x, << * <x». Let x(t) be a Markov 
process, and suppose we are given the initial distribution 


4 1%, f6) = two(s) (fen) de = 1) (4.97) 


. 

of the values x(t). We are interested in the time t,, which it takes 
the process x(t) to first reach the boundaries x = x, or * = xg, 
This so-called first passage takes place at times which vary from 
realization to realization, so that the first-passage time 


: Try = by» _ ty (4.98) 
lis a random variable. 
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We now show how to calculate the mean first-passage time 
<Typ>» confining ourselves to the case where the coefficients K, and 
K, in the Fokker-Pianck equation (4.33) are independent of time. 
Excluding from consideration any realization of the random process 
x(t) as soon as it takes the boundary values x, or x, for the first time, 
we describe the remaining realizations by a probability density 
w(x, t) such that 

4P = w(x, t) 4x + O[(4x)*] (4.99) 


is the probability that at time t, the process x(t) takes a value in the 
interval [x, x + 4x] without ever having reached the boundary 
during the entire time interval [t), t]. Then, the integral 


w(t) = | w(x, t) dx (4.100) 


gives the probability that x(t) never reaches the boundary during the 
time interval [t,, ¢]. Initially, when no realization has yet managed 
to reach the boundary, the probability density w(x, t) in (4.99) is 
the same as the original density (4.97), so that 


W(t?) =1. (4.101) 


At subsequent times, the normalization condition is no longer 
valid, since more and more realizations are excluded from con- 
sideration as a result of having reached the boundary. Sooner or 
later, all the trajectories arrive at the boundary, and hence 


Wo) =0. (4.102) 


Inside the interval [x,, x,], the behavior of the probability density 
w(x, t) is described by the usual Fokker-Planck equation (4.33) 
[or by the conservation equation (4.35)], since trajectories cannot 
terminate inside [x,, x,]. In fact, trajectories are excluded from 
consideration only when the boundary is reached, and there is a 
nonzero probability current at the boundary, corresponding to a 
flow of “representative points” which are ‘absorbed’ by the 
boundary. Therefore, the boundary conditions have to be altered 
in a basic way, as now described. We have already noted that 


SEC. 6] MARKOV PROCESSES AND RELATED PROCESSES 81 


the derivative dx(t)/dt of a Markov process has infinite variance. 
This means that the instantaneous velocity of a ‘representative 
point’”’ (i.e, a point describing a trajectory of the process) is 
ir-finite. However, only a finite.displacement occurs in a finite time, 
which is explained by the fact the velocity of the representative 
point changes its sign with “infinite frequency,” while the point 
mhoves in both directions. Thus, if the random function takes the 
value x at time t, then in the very recent past it must have taken 
values both larger than x and smailer than x. Therefore, at time 
t > t,, there are practically no trajectories near the boundary 
which have not yet touched the boundary. But it is just these 
trajectories which are described by the probability density w(x, £), 
and hence w(x, t) vanishes on the boundary: 


Wx,,t) = w(x,t)=0 (t >t). (4.103) 


Zero boundary conditions like these are typical of problems 
involving first-passage times. 

The initial and boundary conditions (4.97) and (4.103) uniquely 
determine w(x, t) as a solution of the Fokker-Planck equation (4.33). 
After calculating w(x, t), we can find the probability 


Wty) — W(t) = 1 — Wt) (4.104) 


that the boundary is first reached during the time interval [to, ¢]. 
Nifferentiating (4.104), we obtain the probability density of the 
first-passage time (4.98): 


dW 
w(77,) = — at (to + 775) - (4.105) 


Using (4.105), we find that the mean first-passage time is 

ry 0 0 

: <Typ> i J Trp IW (ly + Ty») = | W(t) dt ’ (4.106) 
Typ=0 to 


where we have integrated by parts and used (4.102). If we choose a 
ixed coordinate value x, as the initial condition, so that 


) W(x) = 5(x — Xp), 


82 GENERAL THEORY OF RANDOM PROCESSES [cH. 4 


then w(x, t) is just the transition probability p,,(x, x9), which 
satisfies equation (4.36) with the boundary condition 


Pra (¥%%) =O for x = x, x. (4.107) 


Moreover, (4.107) holds for x) = %,, *,, since then the boundary is 
reached from the very outset, and all realizations are excluded from 
consideration, The transition probability also satisfies the 
Kolmogorov equation (4.39). 

Next, consider the mean first-passage time 


M(%) = Cr19) = | W(t 0) dt, (4.108) 
regarded as a function of the initial value x», where 
W(t, x9) = J * Puug(®s Xo) dx . (4.109) 


To find an equation satisfied by M(xp), we integrate the Kolmogorov 
equation (4.39) with respect to x from x, to x,. Using (4.109), we 
find that 

ow ow 1 aw 

ap = Kalo) Bxy + 2 Salo) Gal (4.110) 
Then, integrating (4.110) with respect to ¢ from ty to oo, and 
taking into account (4.108), (4.101) and (4.102), we have 


dM | a°*M 
—| = K,(%) Ty +5 K4(%o) dee . (4.111) 
If the initial point x lies at the boundary itself, then the boundary is 
“reached immediately,” and the mean first-passage time is zero. 


Thus, equation (4.111) satisfies the boundary condition 
M(x,) = M(x) = 0. (4.112) 


We are now in a position to write down the solution of (4.111) ina 
general form. This allows us to find the mean first-passage time 
(4.106) for an arbitrary initial distribution (4.97), without having 
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to solve the nonstationary Fokker-Planck equation (4.33). In fact, 
substituting the expression 


w(x, t) = i. Pu (%s Xo) Wo(%o) BXo 


A 
ipto (4.100) and (4.106), we obtain 
\ 2 
<typy = |“ M(%0) ol) 2%, (4.113) 
my 


after using (4.108) and (4.109), Equation (4.113) is the generaliza- 
tion of (4.108) to the case of an arbitrary intial distribution w(x). 
For example, suppose that K, = 0, K, = const, Then, the solution 
of (4.111) is 


M(%) = — - ab + Cie + Cr, 
3 
which becomes 
l 
M(x) = Bae (% — *1) (¥2 — Xo) (4.114) 


after using the boundary conditions (4.112). Of course, in a more 
general case, the result is more complicated, but it can always be 
expressed in terms of quadratures. Another approach to problems 
involving first-passage times is given in Vol. II (Chap. 1, Sec. 3, 
and Chap. 2). 


) 
J 


‘7. Replacement of an Actual Process by a Markov Process. 
: A Special Case 

, Any actual process encountered in radio engineering, unlike a 
process which is exactly Markovian, satisfies certain conditions 
involving its smoothness, differentiability, etc. Nevertheless, in 
dome cases we can treat actual processes as if they were processes 
without aftereffect. In this section, we examine under what 
conditions, and with what justification, this can be done. 
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Suppose that the process x(t) is due to the action of another 
process ¢(t) on a system whose behavior can be described by the 
first-order differential equation® 


i = €F[x, &(9], (4.115) 


called the fluctuation equation. Here, ¢ is a small parameter, and 
F is a known function of the arguments x and &. We shali assume 
that the process é(t) describing the external noise perturbations is 
stationary and has a finite correlation time 7,,,- 

We begin by considering the special case where the function F 
does not explicitly depend on x. Then, since F[&(t)] can be regarded 
as a new random function, there is no loss of generality in writing 


& = e€(t), (4.116) 


Choosing t = 0 as the initial time, we can write the solution of 


(4.116) in the form 
x(t) = x(0) + € is E(t’) dt’, (4.117) 


If the initial value x(0) = x) is not random, the cumulants 
k, of x(t) can be obtained by integrating the correlation functions 
R,(t,, ..-, ts)e of the process £(2): 


k, = | is 


{ R(tyy ---y tye Gt, ... dt, (s> 1). (4.118) 
0 


Using the cumulants k,, we can write the characteristic function 


Ss 


O(u, t) = exp > ce kt (4.119) 


of the random increment x(t) — x). Then, taking the time derivative 


¢ More generally, F can also involve the time ¢ explicitly, in which case 
equation (4.115) becomes 


= «F[x(t), €(0, t). 
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of (4.119), we find that the characteristic function satisfies the 
differential equation 


_ ety 
@ = [> “ k,| 6. (4.120) 
Next, we find the corresponding differential equation for the 


probability density 


ts — xy t) = am f Me) Glu, t) du. (4.121) 


Taking the time derivative of both sides of (4.121), and substituting 
(4.120) into the integrand in the right-hand side of the resulting 
equation, we obtain 


33 < k, l \3 p—tu(z—z, 
wb = >, | On fa e~Mlz—%) Ou, t) du. (4.122) 
Since 


ae f Guys ertuteeo) du = (~ 2)" ot f etme @du 


Ox 
Fé] 8 
(5) 
it follows from (4.122) that 
. (—l)'; a 
ps CU koa (4.123) 


Thus, we have obtained a stochastic equation of the type (4.20) 
for a process which is not Markovian. Even in the case of a 
stationary process ¢(2), the coefficients &, depend on the time. 
However, this dependence is appreciable only for time intervals of 
the order of the correlation time (t ~ 7,9,), and the coefficients k, 
are practically constant for t>>7,,,. In fact, according to (4.118) 
and (2.2), we have 


- t t 
= els | " | R(t, typ vony tye dty w». dty_ 
0 0 


0 
= e's | ose f R(7p oeey Ts1)8 dr, eee dr,_) . 
—t —t 
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If we assume that the integrals 


0 
ic sf Ret ves tea) ry drey (8 > 1) (4124) 
are absolutely convergent, it follows that 


k, +é&K, a3 tom, (4.125) 
where 


0 
K, =f wef Riri on ted dro dreg (6 > 1) (4.126) 


are the intensity coefficients of the process &(t). It can be shown that 
the K, are also given by the expressions 


K, = f dee | Rip eoey Ta) dr, dee dr ,-4 
(4.127) 
7 " Feu2 
= 5} d ATo s16 k! pat ces i 
J Ty J, T? J, (71 74-1) 475-1 


where the coefficient K, coincides with (2.8): 
K,=K =2[ Rr)dr. 
0 


According to (4.125), for sufficiently large time intervals, 
equation (4.123) takes the form 


= > OU Ke. (4.128) 


If the parameter ¢ is small, only the lower-order derivatives play an 
important role, and the higher-order derivatives are unimportant. 
If we omit terms corresponding to s > 2, equation (4.128) turns 
into the Fokker-Pianck equation. 

Ofcourse, the fact that the probability density w obeys a 
stochastic equation does not mean that x(t) is a Markov process. 
For this to be the case, one would have to be able to express the 
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multidimensional distributions as products of the transition 

probabilities, after writing the transition probability itself in terms 

of the probability density (4.121), 1.e., 

PAX, %q) = W(x — Xq,7). 

If a multidimensional distribution factors into a product of such 

functions, this means that the random increments 

XY a x(t,) = x(0), X¢ = x(t.) — x(t), coe (0 < fy < ty < ove) ’ 
(4.129) 


corresponding to successive time intervals, are independent of each 
other. Therefore, the Markov condition is equivalent to the con- 
dition that the increments (4.129) be independent. In an actual 
case, however, where the correlation time is finite, there ts correla- 
tion between the increments (4.129). 

For example, consider the increments 


X= x(n) — 0) =e feat, 


X, =x +72) — a(n) = € f ni £(t) dt , (4.130) 


corresponding to the adjacent intervals [0,7,) and [r,, 7, + Tel]. 
The covariance (or cross correlation) of X, and X, ts given by 


x itt, a1] Tt 
K[X, X,] = J . f k(t, — t,) dt, dt, = | i . R(t, — ty) dt, dtp. 
Tz -T) 


(4.131) 
If the integral 


0 < o 
| | k(t, — ty) dt, dt, = ) rh(r) dr (4. 132) 
—o~ 9d 0 
is absolutely convergent, then, according to (4.131), we have 


KIX, X= [shi dr +d, (4.133) 
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where D->0 as 71,7,» ©. It is appropriate to define the corre- 
lation time by the formula 


= [ rk(r)dr, (4.134) 
0 


which is somewhat different from (2.7), but of the same order of 
magnitude. Then, 


2 
K[X,, Xo] > > Kreor (4.135) 


as 71, 7T,-> ©. 
On the other hand, the variance of the random ihcrements 
X,and X, is much greater than (4.135). In fact, 


DX, = 2 [ ; ( "* k(ty — ta) dty dt, = 2e* [ Gps e 
0 0 0 
= &K(r, — Teor) + Dy, (4.136) 
DX, = 2¢ [ Ut DEG IPE a) Das 
v 
where D, ~ 0 as 7, ->™, and D,->0 as r,-> &. It follows from 


(4.135) and (4.136) that for large 7, and 7,, the correlation coefficient 
of the increments X, and X, is approximately 


— Ki%, %2) _ | Teor 4.137) 
o(X,) a(X 2) 2V (T1 —- Teor) (T: — Teor) 


which is small when 


Ty > Teor» Te > Teor - (4. 138) 


Therefore, we can neglect cross correlations, if we only consider 
intervals with lengths much greater than the correlation time. The 
same is true of the higher-order correlations, the only difference 
being that 


ar me at J Tt, dr, [ dro... [RG veey 7-2) ET (s > 2) 
K, Jo 0 0 
(4.139) 
should now be chosen as the correlation time. 
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Thus, for time intervals which are considerably greater than the 
correlation time, the increments (4.130) can be regarded as com- 
pletely independent, and the process x(t) can be regarded as a 
Markov process. This means that the process ¢(t) with correlation 
functions k,(t,, ... t,) can be replaced by a delta-correlated process 
whose correlation functions are 


K S(t, — t,) «..8(t, — ty), (4.140) 


with the same intensity coefficients 


K,= i ~ | Ri(ryy ey Ty) Oty one Egy (4.141) 


-@ 


as the actual process £(f). 


8. Replacement of an Actual Process by a Markov Process. 
The General Case 


We now consider the more general case where no special 
restrictions are imposed on the function appearing in the right- 
hand side of equation (4.115): 


& = eF[x, €(t)] = €F(x, f). (4.142) 


Then the derivation of the stochastic differential equation for the 
one-dimensional probability density is more complicated. Speci- 
fying the value 

X(ty) = Xp (4.143) 


at the initial time ft, we consider the increment 
x(t) — Xo = H (x9) — H (xp, t, ty) > (4.144) 


which now depends in an essential way on the initial value x. To 
find H(x,), we solve equation (4.142) with the initial condition 
(4.143). In fact, we look for H(x,) in the form of an expansion 


H(xq) = €Hy(%9) + 2H (x9) 4- «.. (4.145) 
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Substituting (4.145) into (4.142), we find that 


cH, + @H, +... = «F(%) + = (%o) [eH + ?H, +... 


OF 
-~ 5 as (xq) (eH, + 2H, +...J% +... 


Equating ternis of the same order in «, we obtain 
Fy(xo) = F(x), 

OF 
H,(x9) == rr (%9) Hy(%9) » 


HH (ae) = & (xq) Hefea) +4 S5 (20) HA) 


so that 
: , , 
H,(x,) = | . F(x, t’) dt’ , 


¢ , oF v . ar iid 
H(%) = | tts (wt) J Flew t”) dt" 


[cH. 4 


(4.146) 


(4.147) 


After we have expressed H(x,) in terms of F, to some desired 
accuracy, we consider the statistical characteristics of H(x9). The 


characteristic function of H(x)) can be written in the form 


oa laa) all> CO. CHM) 


(4.148) 


By evaluating the inversion integral, we find the one-dimensional 


probability density 


w(x | x9) = 5. J e~tul2—Z9) [| + > cetH04)] du. 


(4.149) 
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Interchanging the order of integration and summation in (4.149), 
we have : 


W(x | Xy) = [! + > + (- — H(a9))| = i e-tulz—zo) dy 
™ (4.150) 
~ [1+ > 4 (-Z) Head] 8@ — ¥0)- 


Unlike (4.148), this last expression is of a somewhat formal 
character, since the sum of derivatives of the delta function does 
not converge in the ordinary sense. To arrive at expressions that 
converge in the ordinary sense, we have to integrate the product of 
(4.150) and a function f(x,) from some suitable class of functions, 
obtaining 


AG ae ee 
J ms | x) fl) dey = fe) + Dz (— a) KD Se. 
We note that here the function f(x) is first multiplied by (H*(x)), 


and then the product is differentiated. 
It is now convenient to introduce the operator 


L=>4(-2) wey, (4.151) 


where in each term, the operation of multiplication precedes that of 
differentiation. Using L, we can write (4.150) as 


w= (1 + L) d(x — xq). (4.152) 
Differentiating (4.52) with respect to time, we find that 
vy = L8(x — %). (4.153) 


Using (4.152), we can write the last equation in the form 


é=L(1+Ly'w, (4.154) 
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¥ 


where the operator 
(+Ly'=1-L+L?—-L? +... (4.155) 


is the inverse of the operator 1 + L. 

Equation (4.154) is the stochastic equation we are looking 
for, and the operator L(1 + L)-! appearing in (4.154) is given in 
terms of the expansions (4.145), (4.151) and (4.155), which are all 
power series in the small parameter «, since H and hence L are of 
order ¢. Substituting these expansions into L(1 + L)-1 and grouping 
terms involving the same power of «, we obtain an expansion for 
Ld + L)-!, whose first few terms will be explicitly calculated 
below. As the time interval ¢ — ¢, is increased, the convergence 
of the series (4.145), (4.151) and (4.155) becomes worse. However, 
the convergence of the series for La + L)-! does not become worse; 
instead, this series converges to a limiting series which is indepen- 
dent of the initial time ¢,. 

We now find the terms of the series for L(1 + L)-' which are of 
order ¢ and «%. Then, in all intermediate calculations, we need only 
retain those terms which are required for this purpose. According 


to (4.145), (4.151) and (4.155), we have 
b= (-Qjn+(- Sf amy +06 
as ( oe 2) éeH, + &H,> + (- <) (PH, Hy + O(e) 


and 


(I+ Ly =1-L+0@)=1~(-Z) D> + OF) 


=1~«(—2) cap + 0). 


Multiplying L and (1 + L)-?, we obtain 


: oS @? 

Ll +Lyt = —e x (Hy + eH) + aa <A, H,> 
Bt ota 3 (4.156) 
—é& = Ay = <A> + O18). 
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In the last term of (5.156), we interchange the order of the operators 
ajax and <H,(x)>. In fact, using the identity 


Aye) 2 — 2 He Fe] - BED fay, 


we easily see that 


(H(2)> 2 = 2 cya - SEEDY, (4.157) 


and hence 


oH, 
= Sip = > = Say <H,) — mA a 


Finally, taking (4.146) into account, we can write the operator 


(4.156) as 

a a. [dF a 

Li tly? = ) —e# K [55> + #5 KIF, H,] + 0(¢). 
(4.158) 


By neglecting terms of order e? and higher in (4.158), we find that 
equation (4.154) reduces to the Fokker-Planck equation (4.33), 
where 


K(x) = e{F(x)) + &K | 


OF (x) ,H(x)l, 
a (4.159) 
Kx) = 2eK[F(s), Wyo), 


or, because of (4.147), 


OF (x) 
Ox 


K(x) = «<F@) + 2 7 K|  F,(x)| ae, 


K(x) =2¢ [' KEF@.). Ftd! = 28 [KF Feel dr, 


where 


F(x) = Fax,t+r). 
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As t — t, -> ~, we see that 


K,(x) + K(x) , K,(x)—> M(x) + 4 K(x), (4.160) 
where 
M(x) = «<F(x)> 


K(x) = 22 [K[F(s), F(a)] dr, (4.161) 
K'(x) = 4é iN K [Ae ; F,(x)| ‘ 


For time intervals ¢ — tj, which are considerably greater than the 
correlation time 7,,, of the function F[x, €(t)], or of the function é(t), 
equation (4.154) is practically the same as the Fokker-Planck 
equation 


o=—F{[Me)+ EK] ol + 5K) me]. 4.162) 


By analogy with (4.134), it is appropriate to define the correlation 
time by the formula 


: =Jef |7|K[F, Fy] de 4.163) 
a as 7g - T r’t . (4. 


For time intervals t, — to, t, — t,,... which greatly exceed the 
correlation time (4.163), the multidimensional probability distri- 
butions factor into products of transition probabilities (4.149) 
satisfying equation (4.162). The reason for this is essentially the 
same as in the preceding sections, but the proof is more com- 
plicated. Then, the random process x(t) will be approximately 
Markovian, and, to this extent, all that has been said about 
Markov processes will apply to x(t). 
According to (4.21) and (4.16), the coefficient 


K(x) = M(x) + 4K'(x) (4.164) 
is the ‘‘average derivative”’ 


tim Sx? = Cay = CF I(t), COD = Ki@)- 


T 
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It follows that if we wish to calculate the mean value of 
eF [x(t), €(t)], we have to allow for correlation between x(t) 
and £é(#). According to (4.161), as a result of this correlation, 
<eF [x(t), €(t)]> differs from the mean value 


M(x) = Flex, ED), (4.165) 


calculated without regard for correlation between x(t) and &(t), by 
the quantity’ 


é f. K [E- F,] d= + O). (4.166) 


Similarly, to calculate the mean value of any other function 
G[x(t), €(t)] of the random process x(t) satisfying equation (4.142), 
subject to the condition x(t,) = x,, we have to allow for correlation 
between x(t) and ¢(t) by using the formula 


(Gtx) AID = Clan Ad +e f_K [SED Fen] ar + 0 
(4.167) 


where the first term in the right-hand side is the mean value 
calculated without regard for correlation between x(t) and €(f). 

Among the possible functions F(x, €), an important position is 
occupied by those which satisfy the relation 


OF oF, 
im oF] a=f K[RGS ie (4.168) 


which can be regarded as a kind of time-symmetry condition (i.e., 
symmetry with respect to the time reflection t-» —t#). If (4.168) 
holds, then K’(x) can be obtained by differentiating K(x): 


K'(x) = ny, (4.169) 


7 When calculating the averages in (4.165) and (4.166), the argument x is 
regarded as a fixed quantity, rather than as a function of time. 
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Then equation (4.162) can be written in the form 

‘ Q 1 a ow 6) 

to = ~ = [Mo] +4 [KS + 5 (kw). (4.170) 


If M and K are independent of time, (4.170) gives the stationary 
distribution 


__C 7 M(y) 
wads) = aes |? | rae dy\, (4.171) 


which is equivalent to (4.49), because of (4.160) and (4.169), and is 
valid under the same assumptions. 
In particular, the relations (4.168) and (4.169) hold in the special 
case where 
& = €F[x, £(t)] = f(*) + 8(*) &(t) ; (4.172) 


here, f(x) and g(x) are known functions, while é(t) is a stationary 
random perturbation with zero mean value and intensity coefficient 


K, 1.€., 


<&t)) =0, 2 [ {£6 dt =k. (4.173) 
In this case, according to (4.161) 
M(x) = f(x), K(x) = g(x), (4.174) 


and the stationary distribution (4.171) becomes 


w4(*) = w exp {= a5 dy}. (4.175) 

We can also solve the inverse problem, which consists in finding 
a fluctuation equation (4.142) for a random process satisfying a 
given Fokker-Planck equation. Of course, in general this problem 
does not have a unique solution. However, the solution will be 
unique if we restrict ourselves to equations of the type (4.172), 
containing a Gaussian delta-correlated random process é,(t) with 
zero mean and unit intensity: 


Kft) =0, <boborr = 87), 9 (kK = 1). (4-176) 
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Then, using (4.164), (4.169) and (4.174), we find that 


ge) = Va), fe) = Kya) — a) 


(4.177) 
Therefore, the arbitrary Fokker-Planck equation (4.33) corresponds 
to the fluctuation equation 


K(x) — } *2@) . VG) ett). (4.178) 


We can associate a fluctuation equation of the form (4.178) with a 
more complicated fluctuation equation (4.142) which has the 
same Fokker-Planck equation; two such equations are said to be 
(stochastically) equivalent. This method of simplifying the equation 
of a random process by replacing it by an equivalent equation turns 
out to be useful in solving certain problems, and wiil be used later. 

Finally, we consider briefly the error committed when we 
discard terms of order <? and higher in (4.154) and (4.158). 
Retaining all the terms would lead us to a stochastic equation (4.20), 
where each coefficient K,(x) can be expressed in terms of F as a 
power series in the parameter e (beginning with terms of order e°), 
whose terms can be found by applying the method used above. We 
are mainly interested in the limiting values of the coefficients 
K,(*) which come into play when the influence of the initial time ¢, 
vanishes, i.e, when t — ty > Teor Thus, we go one step further 
in the direction of making formula (4.159) more precise, by retaining 
terms of order e? in our calculations and neglecting terms of higher 
orders. This leads® to the following formulas: 


K,=M+44K'+ sf. fx oF Fy Fal dr do 
+e [a f_{K ee F,] + (S)« (= 
— Fk (=, oe do, 


8 The details are given in the Supplement, p. 126. 
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Kee Ka 26 [ f_ K [= FF] dr do 
+22 f ar [" JK[AE : F,] + HE )KIF, F] 


~. (F,)K [F, =I| do, 


0 0 
Ke i i K[F, F,, F 4] dr do, 
Re 5 05 (4.179) 


Here M, K’ and K denote the expressions (4.161) previously 
found, of orders « and e?. In the spectal case where the fluctuation 
equation has the form (4.172), the formulas (4.179) can be con- 
siderably simplified and reduce to 


afi te Ser lez +o ES (2), 


Ky = weg? + nage wa Deg? < im (4.180) 
Ky = nog, 


where 


m= 2 eb ar, 
m= 3) [KLE by bd drdo, 


c= li lr] <€£.) dr. 


We now examine the relative size of the terms appearing in 
(4.179). Taking account of (4.161), we can make the following 
order-of-magnitude estimates: 


Diss) OF ,, 
q* ~ & = Freer s 

OF oF \3 (4.181) 
K,-M—j{K ~eé7GF ‘hor + (= —) Fr or 
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Provided that 


oF OF \* 
(ar) S (Ge) » 
we can conclude from (4.181) that the terms in e* in the expansion of 
the coefficient K, are smaller by a factor «(@F/0x)7,,, than the 
term in e?. Similarly, we can convince ourselves that this factor 
describes the relative size of the correction K, — K, compared 
with K,. In fact, more generally, the corrections due to higher-order 
approximations to the coefficients K,, involving higher powers of 
the parameter e, have relative sizes described by an appropriate 
power of the ratio 


Loop OF 
ty we By Teer’ (4.182) 
Here, 
~1 
ro ~ [eo (4.183) 


is a time constant, which can be called the relaxation time. A 
necessary condition for the effectiveness of the methods developed 
in this section is that the inequality 


OF 
€ Fy Teor < 1 (4.184) 
should hold. 


We introduced the small parameter ¢ in order to make it clearer 
how to construct successive approximations in determining the 
coefficients of the stochastic equation. However, in all actual cases, 
« occurs in combination with the function F. In fact, in solving 
practical problems, one knows the whole right-hand side eF = ® 
of equation (4.115), and there is no need to introduce e« at all. 
Thus, the quantity (@@/@x) 7,,, is actually the parameter which 
determines the convergence of the successive approximations. 


100 GENERAL THEORY OF RANDOM PROCESSES [cu. 4 
9. Systems of Fluctuation Equations Involving Several Processes 

All the considerations of the preceding section can be imme- 
diately generalized to the case of a system of equations 


Hy = EF (my, «.., Xp, t) = EF, (x, t) ’ 


acs sie ; : (4.185) 
Xp = EF, (x, «+, Xp, t) = EF, (x, 2), 


which define the random processes ~,(2), ..., x,(t). For fixed values 
of x), ..., Xp, the expressions F(x, 2), ..., F(x, #) are known random 
functions of time. Choosing the initial values 


X(t) =X (= Les P), (4.186) 


we write the solution x,(1 </ <p) of the system (4.185) as an 
expansion 


t 
x(t) — tie = Hr(e) + € | Fulton ty) dts 
(4.187) 
Rot OF 
+2 é { ; dt, = - (9, t) _Fn(to ta) dtg +... 


The increments x, — x19, ..., %» — "po have the joint characteristic 
function 


(exp bm u(x, — u)}) =1+ » tu,(H ,(x9)> 
(4.188) 
+ im init (0) Halo) + 


By analogy with (4.150), we can write the multidimensional 
probability density, which is the inverse Fourier transform of 
(4.188), in the form 


W(X — XyQy «vey Xp — Ayo) = (1 +L) S(x, — x49)... 8(%_p — Xp), (4.189) 
where 


ele de) Hila + 7% ts as) (— sez) <Hhes easier 
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It follows at once from (4.189) that the probability density 
W(X, — X49, ++) Xp — Xpo) Satisfies the equation 


w= Lil +Lyw. (4.191) 


We must now calculate the operator L(1 + L)-!. Substituting 
(4.187) into (4.190), we obtain 


es D2 ao if (F(x, ty)> dt, 
aio f. dt, f y (Se OF G4) n(X, tr) dt,| (4.192) 


Pa 
a} = 2 Fain Y Sia { a CF (x, t)) Fn(%, te)> dt, dtz + Ofe*) , 


b= ae [ts t)> ef Le (x, t) F(x, t’)) ae’ 
. (4.193) 
ae “Sate { : (F(x, t) Fiq(x, t’)> dt’ + O(€). 


Just as before, the operator £(1 + L)-! differs from (4.193) only by 
having the correlation functions 


K [$2 ,Fals  KiFo Fal 


in place of the averages 
oF, 
(Ft Fm) FiFn) 
As a result, equation (4.191) becomes 


v= = (Po +e JK [Gt Fae] dr) 


e (4.194) 
42 > aaa in K[Fy, Fn] dr w , 
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except for terms of order e? and higher. Here, in calculating the 
averages <F,> and the correlation functions in the integrands, the 
arguments x;,...,#, are regarded as fixed. For t — ty S 7oo,, the 
lower limit t, — ¢ can be replaced by —o. 

We see that to within the given accuracy, the equations (4.185) 
are equivalent to the multidimensional Fokker-Planck equation 
(4.79). We now show that, conversely, every Fokker-Planck 
equation can be replaced by a system of fluctuation equations 


%, = f,(x) + Bil) Em(t)  (L= ly ef), (4.195) 


where the functions f,(x) and g,,,(x) are suitably chosen. Here, 
£,(t), .--. €,(t) are independent, Gaussian, delta-correlated random 
functions with zero mean values and unit intensities: 


<ép = 0 1 <L:Eme> == 81m9(7) - (4.196) 


Applying formula (4.194) to the equations (4.195), using (4.196), 
and comparing the result with (4.79), we obtain 


lw By 
2 4 OXm 


K(x) = f(x) at Emi» 
(4.197) 


Kim(*) = > 81,(*) &mi(*) 7 


We can always go from the Fokker-Planck equation to the system 
(4.195), provided that the relations (4.197) can be solved for 
the functions g,,,(x) and f,(x). But this can always be done, as 
we now show. The matrix K =|| K,,,{| is symmetric and 
positive definite (more precisely, nonnegative definite), and hence 
it follows from familiar results of linear algebra that there exists a 
real, symmetric, nonnegative definite matrix G = || g;,, || which 
ts the square root of K, 1.e., 


G = K?*/?, G-=K, D> fu8im = Kin- (4.198) 
$ 
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In fact, because of the symmetry condition g;,, = mj (4-198) is the 
same as the second of the equations (4.197). It follows that the 
functions g,,, and hence the functions 


lw 9%; 
fx) = Kix) — 5 ae Bin (4.199) 
can always be determined. 
To calculate || gi, ||, we can use the orthogonal matrix 
iJ = || 44, || which reduces K to diagonal form, i.e., 


UKU-} = || K%,,|| (K° >0). (4.200) 


In terms of U, we have 
G = U+|| VKes,,|| U. (4.201) 
‘Using the u,,, we can write the equations (4.195) in the form 


> Uk) = > taf + VKon(t), (4.202) 


q 


‘where they have been solved for the “fluctuations.” Here, 
ed > Urb, (t = |,..., p) 
4 


ire a system of random functions just like the é(¢), and in fact 
satisfy the same conditions (4.196). 

In the diffusion approximation (which corresponds to neglecting 
cerms of orders e, <*,... in the stochastic equation), any system 
of equations (4.185) can be “simplified,” in the sense of being 
replaced by an equivalent simpler system (4.195) or (4.202). 


10. Gaussian Markov Processes 


Within the class of Markov processes, the simplest are those 
which are described by a Fokker-Planck equation (4.79) where the 
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coefficients K,,, are constants and the coefficients K,(x) are linear 
functions: 


K(x) = — > Bimtm +4 (L= Ly oe, f)- (4.203) 


According to (4.195) and (4.197), this Fokker-Planck equation is 
equivalent to the fluctuation equations 


+ D> Prom = nit), (4.204) 
where 


nit) = by + >) Simém(t) (4.205) 


are Gaussian random processes such that 


<m> = b, 
K[n,, mt] =a Kim9(7) ° 


The system of linear equations (4.204) can always be solved, and 
as a result, x,(t), ..., *,(t) can be expressed linearly in terms of the 
known random processes 7,(t), -... 7p(t). Therefore, the processes 
x,(t), ..-, ¥)(t) are also Gaussian, and their statistical characteristics 
can be easily determined. 

In order to calculate the cross-spectral densities of the processes 
Xy, ++) Xp, we replace the differentiation operator in (4.204) by 
iw and find the inverse of the matrix 


(4.206) 


| Bim “t- 108 ten | . 


Let A(éw) be the determinant of this matrix, and let 4,,,(¢w) be the 
cofactor of the element in row / and column m, so that 


A imn(t) . 


Bim + ibm I~? = || Ay (4.207) 


and 


Arias 
4= i > fee (4.208) 
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Using (4.208), we find that the cross-spectral density of the 
f rocesses x; and x, is 


S[x;, Xk w] or > 
l 


om 


Ay(iw) Aj,,(tu) 
Aw) d*@w) 


S[ny mie]. (4.209) 


But because of (4.206), (2.54) and (2.12), we have 
Slo mie] = 2K + 417b,b,,8(w) - 


In particular, setting j = k, we find that the spectral density of the 
process x;(t) [centered at its mean] is given by the expression 


S[x; — <x; w] = 2 | A(iw) |-* ) Ap(iw) 47,,(@) Kim - (4.210) 
I,m 
lfere, the numerator and denominator contain polynomials in 
«2, A process with a spectral density of this kind is said to have 
a rational spectral density. 

Thus, we have shown that Gaussian Markov processes have 
rational spectral densities. The converse assertion is also true, i.e., 
every Gaussian process with a rational spectral density can be 
represented as a component of a multidimensional Markov process. 
This fact is important from a theoretical point of view, because the 
spectral density of any actual process can always be approximated 
to any desired accuracy by a rational function. Therefore, an 
actual Gaussian process which is non-Markovian can be represented 
approximately as the component of a multidimensional Markov 
‘yrocess, and by increasing the number of components of the 
Markov process, the approximation can be made more accurate. 
This question will be discussed further at the end of Sec. 12. 

We now consider in more detail the simplest process x(t) with a 
‘ational spectral density. This process, which is simultaneously 
stationary, Gaussian and Markovian, plays an important role in 
ioise theory, and will be called the exponentially correlated process, 
1 designation whose meaning will be apparent from the formula 
Jerived below for the correlation function of x(t). According to 
(4,204), for a one-dimensional process we have 


+ px=7n, (4.211) 
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where 
y= 5, Kn, 7.) = Ka(z). 


In this case, formulas (4.208) and (4.210) become 


$ += 


B+ tw’ 


S[x — Gia] = 77 “ 


om = poe (4.212) 


To study what happens when the process x(t) is “‘switched on,” 
we impose the initial condition 


x(t) = % - (4.213) 
Then equation (4.211) has the solution 


t 
x(t) = xge~Alt-te) I, e At-) n(s) ds. (4.214) 


Using (4.214), we find the following expressions for the mean yalue 
and the correlation function of x(t): 


t 
<x(t)»> = Xe Alt-tod 4- Ii, e-Alt-9 b ds = 3 + (x0 ane 3) e7 Alt—te) ; 


— (4.215) 
K[ x(t), {¢2)] = [of “eM those) K8(5, — 54) doy dy 
to~ te 
If to be explicit, we choose t, — t, > 0, then 
th K 
K[x(t,), x(t,)] = K | e-BltyHty-29) Jy =z S [e-Allat) — @-Blty tty Bho] 
te 2p 


In general, we have 
K[ x(t), a(t] = Za [eel — errete eto], (4.216) 
Ast — f, increases, the “stationary” correlation function 


k(t) = o° R(t), = : R(r) = eA ltl, (4.217) 


BI 
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corresponding to the spectral density (4.212), becomes ‘“‘estab- 
jished.”” Moreover, according to (4.216), after the process is 
switched on and while it is reaching equilibrium, its variance is 
given by the formula 


Dx(t) = o7[1 — Rt — t))]. (4.218) 


Since the process x(t) is Gaussian, just like (t), the mean value 
(4.215) and variance (4.218) completely specify the one-dimensional 
probability density 
I x — m — (%» — m) R}? 
W(x) = Pr_.(%, Xo) = oT Vil = BB eXP ) — ae , 

(4.219) 
where 


R = R(t — to), ma. 

The probability density w(x) corresponds to the condition (4.213), 
and hence is just the Markov transition probability (4.4), in terms 
of which the arbitrary multidimensional distribution (4.6) can be 
written if we know the initial distribution. 

The formulas (4.215) and (4.216) are special cases of the general 
formulas (3.26). Therefore, the distribution (4.219) gives the 
conditional density w[x(t) | x] for any stationary Gaussian process. 
However, (4.219) can be used to construct the multidimensional 
distributions only in the case of the exponentially correlated 
process, for which R(7) = e-4'"!, 


11. Second-Order Fiuctuation Equations. 
Solutions of Special Cases 
of the Two-Dimensional Fokker-Planck Equation 


In many problems, the noise process x(t) under consideration 
satisfies a second-order fluctuation equation 


# = Flx, 4, &(t)), (4.220) 
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where £(t) is a random process with a small correlation time 7¢o,. 
Introducing the notation 
t 1 


i= ’ your = 
Teor Teor 


we can write (4.220) as 


or as a system 


i = Toor F(%, y» £) . 


The small correlation time 7,,, in the right-hand side of (4.221) 
plays the role of the small parameter « which appears in the right- 
hand side of (4.185) and allows us to go from the system of 
fluctuation equations (4.185) to the Fokker-Planck equation 
(4.194). In terms of the original symbols x and t = 7,,,¢, the 
Fokker-Planck equation takes the form 


w(x, 2) = -sfu-% (F> +f_K [=F] dr) wo} 


oe (4.222) 
+35 [KF Fl dew}. 
We now analyze some special cases of (4.220). 
11.1. Let the fluctuation equation have the form 
pit + & — f(x) = Kt), (4.223) 


to which we can reduce the somewhat more general equation 


ai + bt = f(x) + a(2). (4.224) 
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Moreover, let &(#) be a delta-correlated random process with zero 
mean, i.e., 


E> =0, <8) = xd(r)- 
‘Then if we write (4.223) as a systein 


H 


pe = ys 
By = —7 + fe) + £0), 
the corresponding Fokker-Planck equation becomes 


‘ Ly Ow ow 1 5a « Ow 
w(x, y) = a [> + f(x) al + re [5 O™) + 5 BA » (4.225) 


from which we can derive the following results: 
1]. It is easily verified that the stationary distribution corre- 


sponding to (4.225) is® 
Y,2¢ 
W(x, y) = Cexp ar So am { f(z) ds| . (4.226) 
since when w has the form (4.226), each of the terms in brackets in 


(4.225) vanishes. It follows from (4.226) that the stationary 
distribution of x is 


(2) = C exp 2 | fe) dal, (4.227) 


which is independent of ,» and identical with the distribution of 
the one-dimensional Markov process described by the equation 


% = f(x) + &(t). 


In fact, this equation can be obtained either by setting » = 0 in 
(4.223) or by setting g = | in (4.172). Therefore, the stationary 
distribution (4.227) is of course the same as (4.175) when g = 1. 


* In this equation and the next two, C denotes the normalization constant, 
which is suitably chosen in each case, 
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The stationary distributions (4.226) and (4.227) exist subject to the 
same restrictions on the function f(x) as in the case of a one- 
dimensional Markov process. For a fluctuation equation of the 
form (4.224), the stationary solution (4.226) is replaced by 


0,,(x, 4) = Cexp| — 2 rs > [f f(z) dx + <g> x| , (4.228) 


where 


«= Kis). s(é)] dr. 


2. To find the nonstationary solution of equation (4.225), we 
write w(x, y) in the form of an expansion 


w(x, y) = > Tnn(t) Xm(*) Yay) (4.229) 
mn=0 


in terms of the eigenfunctions X,,(x), Ya(y) which satisfy the 
equations 


* a An) z 7 [£(%) Xm(@)] + AmXm(x) =O, (4.230) 
5A EE DMOM TMK) =0, (4.231) 


and the orthonormality conditions [cf. (4.56)] 
ax dy 
J Kul) Xa) Gy = Seer J Yat) Fa) PE = Bae 
Substituting (4.229) into (4.225), we obtain 
DT nnXm¥n = —t dy TranlXe¥¥n +f(*) Xm¥al — DY) MnTmnXm¥n- 
Using the relation 


- Xnv¥n +408) Xm¥a = An [ Xm de Yn — MXm | Ya dy, 
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which follows from (4.230) and (4.231), we can transform the basic 
equation into 
; 


L)(u*Fnn + eT an) Xm¥n = HD TealdaXe [ Yq dy — Ax [ Xn de Ye). 
mn kn 


(4.232) 


. We now expand X,, and [X,dx in terms of the eigenfunctions X,, 
by writing 


Xj(~) _ > Xn(*) amk 


(4.233) 
J X{*) dx = a X,,(*) Bink » 
where the expansion coefficients 
dx 
mk | Xm Xe ’ 
(4.234) 


han fulfna)e 


obviously form matrices which are inverses of each other, i.e., 
D>, Amu kt = Smt 
k 


We also introduce similar coefficients for the functions Y,(y) and 
JY.(y)dy. In fact, since equation (4.231) is the same as equation 
(4.68), it has the same eigenvalues A, = n and eigenfunctions 


Y,(y) = jo. —— Fin) (/2.5) (4.235) 


[cf. (4.69)]. It follows that 
; 20-—— 
Y,) = ne n+ 1 Vruly ) ’ 


(4.236) 


J %) dy = > 97 Y,1(9) 


Substituting (4.233) and (4.236) into (4.232), replacing A, by , and 
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equating coefficients of identical functions X,,Y,, we obtain the 
system of equations 


Tn + ®T mn = He f Zz VAAL DY dmb nt 
k 
(4.237) 


2. 
— Ba, — VI 2, bateTean-s (m,n = 0, 1,2,..), 


which is equivalent to the two-dimensional Fokker-Planck equation. 

When the parameter yp is sufficiently small, we can solve (4.237) 
by a perturbation method, which goes as follows: Setting » = 0 in 
(4.237), we obtain 


PT mo = ye yy Amt] ka » 
k 


where here (and from now on), p denotes the differentiation 
operator 9/dt. But T,, can be found by setting n = | in (4.237); 


2 
Tn = 1+ aoe eee 2 ml rg — T4 pp ar VES bdTe - (4.238) 


Then, to find T;,, we set » = 2 in (4.237), obtaining 


V2p 2 
fae eae 2 44753 — 24+ pp 2 OATa 


which we then substitute in the right-hand side of (4.238). This 
procedure leads to the following expansion in even powers of the 
small parameter p: 


Hm 
pp / eae aaa 1+ p> Tino 


2p ‘ 
=F (1 + ep)? (2 + pp) 2, AmAD AT ig + O(n). 


(4.239) 
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After deciding upon a given accuracy, we can discard all terms of 
higher orders, while simplifying the remaining terms somewhat. 
Thus, in the second approximation, we have 


Am 
le a Ta Tmo = p* 2 AmirceriT ig + Oe') - 


Transforming the expression in brackets, we obtain 


P+ Am(L — wip)! = p + Am — H?Amp + O(u4) 
= (1 — pA.) [2 + Am (L — 2?An)t] + O(e*) 
= (1 — p*A,,) [2 + Am + #*A2) + O(pt). 


Therefore, to the same accuracy as (4.239), we can write 


Tmo = — (Am + #22.) Tmo + 1? D, @micdeParri Ty + O(n). (4.240) 
kt 


The equation (4.240) describes how the coefficients T,,,) vary in 
time. If 4 = 0, each coefficient T,,, decreases exponentially with 
‘time constant” A;', just as in the case of one-dimensional Markov 
processes (see Sec 4). Thus, if » = 0, a perturbation which has 
the form of an eigenfunction X,,(x) decays without changing its 
shape. However, if » = 0, the perturbation still falls off exponen- 
tially, but it has a somewhat different form. We now illustrate this 
situation by examining the form of the perturbation which falls off 
most slowly (with time dependence given by the factor e”*'), and 
which coincides with the first eigenfunction X,(x) when p = 0. 


Setting 0/2 = p, in (4.240), we find that 


Tmo Fe 2y amedebarsT io + Olu‘), — (4.241) 
m k 


pe 
a eee 


for m + 1. Moreover, setting m = 1, we find that the constant p,, 
which determines the rate of decay of the perturbation, is equal to 


Pi = —A — WAL + WPA, > OmpAcPrr + O(n") » (4.242) 
k 
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The one-dimensional probability density w(x) is given by the 
formula 


w(x) = > T moXm(%) 5 (4.243) 


m=a0 


which can be obtained by integrating the expansion (4.229) with 
respect to y. Substituting (4.241) into (4.243), and retaining only 
terms of order »*, we find that the ‘‘nonstationary perturbation’’ 
which falls off most slowly while the stationary distribution is 
being established has the form 


w(a) = Tw [X(8) + ww DES eet X(2)], (4.244) 


and is proportional to the “modified eigenfunction” corresponding 
to X,(x). Similarly, we can find the shapes and time constants of the 
other modified eigenfunctions. 

x It should be noted that equation (4,240) is equivalent to a 
certain differential equation for the one-dimensional probability 
density w(x). To derive this equation, we multiply (4.240) by 
X,,(x) and sum the result over m, as suggested by the form of 
(4.243). In doing this, we take into account the relations 


Y Xn(2) doe = ~ D Xe) Ce, 


© Xal#) AaCm = [/l8) — FG] D Xa) Cm, 


which follow from (4.233) and (4.230). Thus, the coefficients a,,, 
correspond to the differentiation operation, the coefficients },, 
correspond to the integration operation, and the 4,, correspond 
to the operation 


Ox = [se a 2 Fal: 
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Then a simple calculation leads to the differential equation 


we) = 2 [1 — ne) | < 2M)) (4.245) 


where f'(x) = Of(x)/ax. If this equation is regarded as an ordinary 
one-dimensional Fokker-Planck equation, the corresponding func- 
tions K,(x) and K,(x) are just 


K(2) = f(z) — Y#@S'@) — BSS"): 
K,(2) = «{1 — p¥'@)) . 


Naturally, if higher-order terms in » are taken into account, the 
corresponding equation will be more complicated. 

3. If the parameter 7 is not small, another method has to be used 
to find the solution of equation (4,223). Thus, we now consider 
the opposite extreme, i.e., the case where 7 is large. Writing 
1 /p = ¢ and replacing ¢ by t/e, we can transform equation (4.223) 
into the new equation 
& + ex — f(x) = Ve &(t). (4.246) 
] 
If « is small, this equation describes the behavior of a system 
performing nonlinear oscillations under the influence of weak 
frictional forces and weak external fluctuations, It is convenient to 
introduce the ‘‘energy” 


E= 424 u(x), (4.247) 
where 
| ux) =~ f(z) dz 
is the “potential function.”” Multiplying (4.246) by %, we obtain 
E = —ext 4 Ve x&(t). (4.248) 
Thus, from (4,247) and (4.248), we obtain two fluctuation equations 


| * = VIE — u(x), 
E = —2¢[E — u(x)] + V2e[E — u(x)] &(t). 


(4.249) 
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Then, in the usual way, we can associate (4.249) with the following 
Fokker-Planck equation for the probability density w(E, x): 


ti(x, E) = — 2 [VIE u) w) + 22, [(B —u — 4) w] 
(4.250) 
+ «« Aa ((E~u)u), 


where 


«=f eden. 


If « is small, the energy E ts conserved during a large number of 
periods of the oscillations, and moreover, the time which 4+(t) 
spends at the point x is inversely proportional to the velocity 
x = V2(E — u). Therefore, for a fixed value of the energy, the 
conditional probability density is 


ore [EZ — u(x)]-*/ for u(x) < E, 


w(x | E) = forus) SE, 4251) 


Normalizing this distnbution, we can write the two-dimensional 
probability density in the form 

w(x, E) = w(E) w(x | EB) = ——™2)___ (4.252) 

2p'(E) VE — ue) 
Here 
dx 

p(E) = 7 
2 5 nen VE VE — u(x) 


and the integration is over the region R(£) where u(x) < E. 

The distribution (4.252) can also be derived directly from 
equation (4.250). In fact, after a ‘‘quasi-equilibrium distribution” 
has been established, characterized by the fact that w is of order e, tt 
follows from (4.250) that 


2 IVE — u(x) w] = O(), 


(4.253) 
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which when integrated gives 


ees) ae 5 'e 
ar (e). 


Thus, the deviation of w(E, x) from (4.252) is a quantity whose 
order of magnitude is «. 

Substituting (4.252) into (4.250) and integrating with respect to 
x, we obtain the one-dimensional Fokker-Planck equation 


E x 2 
AE) cs [( on 72 Sie ae ae | ey) 


for the energy, where 


AE) ={ VE — W(x) dx 
R(E) 


[cf. (4.253)]. This reduction of the two-dimensional Fokker-Planck 
equation to a one-dimensional equation considerably simplifies the 
problem of investigating nonstationary noise problems. Of course, 
if we take account of (4.252) and (4,247), the stationary solution of 
(4.254) reduces to the solution (4.228) found previously. 


11.2. Next, we consider the somewhat different equation 
| 


% + h(x, #) — f(x) = Ve &X(t). (4.255) 


In this case, the stationary solution of the corresponding Fokker- 
Planck equation can no longer be written so simply in terms of 
quadratures, as in the case of equatiom(4.224). However, for small 
values of the parameter «, we can apply the method just described, 
which, in particular, allows us to find the stationary distribution. 

Thus, multiplying (4.255) by x and introducing the energy 
(4.247) instead of x, we obtain the equations 


—eV2E — u) hx, V2(E — u)] + V2e(E — u) &t). 


. 
3 
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The Fokker-Planck equation corresponding to (4.256) differs from 
(4.250) only by the form of the second term in the right-hand side, 
where instead of 2(E — u), we now have 


VUE — u) h[x, V2(E — u)]. 


Substituting (4.252) into this Fokker-Planck equation and inte- 
grating with respect to x, we obtain the one-dimensional equation 


"a x @ 
iE) = «55 [( Seer - -5) 4] +<$oR ay wo], (4.257) 


where 

WE) = m7 i Als, VE — w)] ae (4.258) 
From (4.258), we can find the stationary distribution in the usual 
way: 


w,(E) = const - '(E) exp|~ = is a aE’ (4.259) 


Using the relation 


_< F(E) = an (4.260) 


to introduce the function F(£), we can write the two-dimensional 
distribution (4.252) in the form 


_ const 


—+r@)|. (4.261) 
Going over to and bearing in mind that dx dE = x dx dx, we have 


Wy(x, x) = x exp } — 2 F F x? + u(e)|| ; (4.262) 
where 


N = fexp|——F [59% + wa)]| de dy. 
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If we integrate (4.262) with respect to x, we find that the stationary 
distribution of the coordinate x is 


W,(x) = x exp — 2 59" + u(x)|| dy. (4.263) 


When A(x, %) = Bx, the functions ¥(£) and ¢(£) are proportional, 
and then the distribution (4,262) coincides with (4,228). 


11.3, Finally, we consider the equation 
pee + [L + pth(x))  — f(x) = e(t), (4.264) 


which is more general than (4.223). The corresponding Fokker- 
Planck equation is 


pits, 9) = — 4 [y 2 +f] 
; : : (4.265) 
+ whe) 5 (ore) + 5 0m) +553. 


where y = px. We shall assume that the parameter » is small, 
exploiting this fact to solve (4.265) by a method resembling that 
‘used on pp. 110-114. 

‘ First, we write the probability density w(x, y) in the form 


w(x, y) = > w(x) Y,(y) , (4.266) 


where the Y,(y) are the eigenfunctions (4.235) of the equation 
4,231). Next, we substitute this expansion into (4.265), using the 
first of the equations (4.236) and the relations 


~ (y¥,) — —Vatl Vn+2Y,,.—Y,, 
By (4.267) 


\j 


yl, af XtVeFT Yan + VEN, 


which follow from (4.231) and (4,236). Then, grouping coefficients 
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of identical eigenfunctions in (4.265), we obtain the system of 
equations 


(n+ np + pa ng = pal vee Stash 
(4.268) 


oa H OW,_1 
“P BA aes +> Bx 


—pvn—1lVn hon.) 


where p = 0/8. This system can be solved by the method of 
successive approximations, If we choose the same accuracy as on 
pp. 112-113, then it is sufficient to retain only the first three 
equations, which correspond to 2 = 0, 1, 2 and have the form 


_ i /2 7) Re l 
m= Sete ra BES oT Weshal L 


(4.269) 
Grtstis gl sass IW Oe a 
2 END 24 pip + pth dx? 
2 V2 wev2h 
a a On — at 
where 
Kk O 


Substituting the second of these equations into the first and 
third, and using the third equation as well, we obtain an equation 
which, to the required degree of accuracy, can be written as 


Q l 


oe? Kk 
pio path oe OE 9 oa te OP) 


2 Ox? 


Transforming (1 + »?p + y?h)-! into 1 — pp — ph, transposing 
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the term —229(0/8x) Dw, to the left-hand side, and dividing both 
sides of the equation by 1 + »7(2/0x) D, we obtain 


Q 3 oe 
pw = 5, Dwg (5 D) Wy + pi 5 Diy 
; : (4.270) 
K 
— ps hDw, — p? i a) 


Then, using the form of the operator D, we combine the second 
and third terms in the right-hand side of (4.270): 


me -(%) = zl? Pa? 


Thus, ae (4.270) can be written in the form 


woe) = 2 Jt — pry’ — ph) [fo + £2] — SZ tes). (4.271) 


Here, we have replaced w (x) by w(x), since w(x) is just the 
one-dimensional probability density of the coordinate x, as can be 
verified by integrating the expansion (4.266) with respect to y. 
‘Thus, we have obtained a one-dimensional Fokker-Planck equation 
which is equivalent to the two-dimensional equation (4.265), to the 
c.egree of accuracy chosen. In particular, using (4.271), we can find 
the stationary distribution 


w(x) = const - exp |= f° f(a) [1 + wAh(z)] dz + w%h(s)!, (4.272) 
{ *1 


which is a generalization of formula (4.227). If the fluctuation 
equation has thesform 


at + W(x) # = f(x) + ee), (4.273) 
then (4.272) becomes 


| wu(s) = const exp | f° f(z) Ha) de + Hs). (4.274) 
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12. Transition from a Markov Process to a Smoothed Process 


The study of Markov processes is particularly appropriate, since 
effective mathematical methods are available for analyzing them. 
However, a certain care must be taken in replacing an actual 
process by a Markov process, since Markov processes have many 
special features, and, in particular, differ from the processes 
encountered in radio engineering by their lack of smoothness. 

For example, suppose that the process y(t) ts described by the 
equation 


y —Siy) = &(). (4.275) 


If the process é(t) has a small correlation time r,., < to, where, 
according to (4.183), 


of \-} 
ro~ ( ay) 
then, with the accuracy discussed in Sec. 8, the probability density 
w(y) satisfies the Fokker-Planck equation (4.162). By writing 


(4.162), we replace the actual process y(t) by a Markov process 
x(t) which satisfies the equation 


& — f(x) = X(t), (4.276) 


where {(t) is a delta-correlated random process with the same 
intensity K as &(t). Since {(t) has infinite variance, it follows from 
(4.276) that the derivative x has infinite variance (i.e., Dz = -), 
which indicates that the function x(t) lacks smoothness and that its 
graph is “jagged.” However, the inductances and capacitances 
always present in the circuits used in radio engineering act to 
smooth the process x(t), so that to obtain a process like those 
encountered in practice, we have to subject x(t) or {(¢) to smoothing 
with a time constant r,,, < ro. [In the case described by (4.275), r,,, 
is of the order of the correlation time of the process &(t).] The 
smoothing operation considerably modifies the behavior of the 
process x(t) over time periods of the order 4t ~71,,, but the 
‘large-scale’ fluctuations (for which At ~7)S>1,,) remain 


skc. 12] MARKOV PROCESSES AND RELATED PROCESSES 123 


almost unchanged. Thus, the results obtained by applying the 
techniques of Markov process theory are valuable only to the 
extent to which they characterize just these “‘large-scale’’ fluc- 
tuations. 

To get an idea of how the operation of smoothing affects a 
Markov process, we consider a specific example of smoothing, 
obtained by setting 

&(t) = f ett (t") dt’. (4.277) 
This corresponds to assuming that &(#) in (4.275) is an exponentially 
correlated random process, satisfying the equation 


tome + € = X(t). (4.278) 
According to (4.275) and (4.278), we have the following second- 
order differential equation for the smoothed process y(t): 
Tom + [1 —Tomf' WII — f(y) = St). (4.279) 
We arrive at approximately the same equation if in (4.276) we 
average x(t) instead of ¢(t), by setting 


t 
ae a + oO tam aft’) dt! , (4.280) 


1.€., 
TnI ty =. (4.281) 


Then (4.276) and (4.280) imply the equation 
Tsm¥ +¥ —S(tomy + y) = S(t), 


which coincides with (4.279) if we expand f(r, + y) in a power 
series in r,,, and neglect terms of second and higher orders in 74. 

Equation (4.279) corresponds to a two-dimensional Markov 
process, and is a special case of equation (4.264), whose solution was 
studied in Sec. 11. Applying formula (4.272), we find that the 
stationary solution of (4.279) is 


wudy) = const exp |e [ fle)dz— Py) —tmf'(){- (4.282) 
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Equation (4.282) shows the extent to which the smoothing perturbs 
the original stationary distribution. 

If f is a linear function, i.e., if f(x) = —fx, then a(t) ts an 
exponentially correlated process, and 


S[L; @] 2K 


ery 


hee) = 55 eP 


(cf. entry 2 of Table 1, p. 25). The smoothed process y(#) has the 
spectral density 


Sie) 2K 
Wiel ae at” Gather 


According to entry 7 of Table 1, y(¢) has the correlation function 
ho) = 55 (L — Bh? fe Pt — Brg Hom), (4.284) 
where 
Dy = 55 (1 + Bram) 
If Br,,,, < 1, we can simplify (4.284) somewhat: 
Ryle) = syle tel — Prog ion) 


The smoothing operation just described is incomplete. In fact, 
(4.279) shows that the variance of the second derivative of y(t) is 
infinite. To eliminate this infinite variance, while remaining within 
the framework of Markov process theory, we would have to 
consider a fluctuation equation of at least the third order. However, 
any random process actually encountered in radio engineering is 
analytic, and all its derivatives are finite with probability 1. There- 
fore, we cannot describe an actual process within the framework of 
Markov process theory, and the more accurately we wish to 
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approxtmate such a process by a Markov process, the more com- 
ponents the latter must have and the higher the order of the 
corresponding fluctnation equation must be. 

- As an example, we consider the analytic random process which 
has the correlation function 


R(t) = o%e-P**4 
and the spectral density 


| Slé] = - Vn ote (4.285) 
(cf. entry 3 of Table 1). Equatton (4.285) can be rewritten in the 
form 


PVP STe: w] = Vn. (4.286) 


If in (4.286) we make a power series expansion of e®’/® and retain 
only the first x + 1 terms, then the process &(t) ts replaced by a 
Markov process &,() and (4.286) becomes 


w? 1 we 

(1 + get + or ae 

In fact, we can represent the first factor in the left-hand side of 

(4.287) in the form P,(tw)P,(—iw), where P,(p) is a suitable 

polynomial, and we can represent the right-hand side of (4.287) as 
the spectral density of a delta-correlated process {(t) by writing 


) Stéai 0) = =. (4.287) 


4 
: Sh 0] = = 


j 
Then (4.286) becomes 
| Pa(tw) ? S[En3 wo) = S[f, oe), 


which is equivalent to the following differential equation of order 2: 


P, ($-) & = Ue). (4.288) 
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This implies that 


a dng, 
fw a peery  pn- 


(i.e., the process €,(#) and its first n — 1 derivatives) form the 
components of an n-dimensional Markov process. Moreover, the 
larger n, the less é,(t) differs from é(t). If n = 1, the process &,(t) 
is an exponentially correlated Markov process satisfying the 
equation 


gata = Ut), 


and 


6181 p= o /r eWFiti , 


The process £,(2), which ts a closer approximation to £(t), has 
the correlation function 


oer) =o/ ya e-*ttlloos br +¢ sin b | =|], (4.289) 
where 


(cf. entry 5 of Table 1), and satisfies the differential equation 


Beer Mitte, 9. t. 
ai B 


Supplement 


x Since the calculations leading to the formulas (4.179) are rather 
complicated, we give some of the details here: It follows from 


(4.151) and (4.145) that 
LUI -L4+B—.)=eA+0B+8C+.., 
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where 
C= — 2 CH + SHH + AHO) — 5-25 LHD 
- i > 2 CHD + (hay ZH] +52 HD 2a HD 
Sy hHD 2H) — 2H A) >. (4.290) 


We write (4,290) in the form 


a a a 
C=—-7atzsr—gaGs: (4.291) 
where C,, C, and C, are functions which can be expressed in terms 
of derivatives and integrals of the function F appearing tn (4.142). 
If we use the relations 


2 iy 2 ay = 3 ty Hp - 2 (42) ip, 
2 thy & ud = & i> <HD 
2 pile +2 CB) cp 
4H) 2 ay = & hw) SH) 
2 ih 2 cy 2 dy = oy SAD SH 
2 2B) cay + tp (44) cy] 


+ 2B) cay + (2) () ay), 
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which are consequences of the operator identity 
7] 
fe) = Zs) — FO. 


Substituting these relations into (4.290), we obtain 


C, = (Hy — (FB) ca ~ Fag) 0 ~ 5 (BEB) CHD 


+ CB cen + 2) (2), 


Cy = (Hal) + (HyHy> ~ SH) (Fh ~ (FL) <P) — Cs (Ht 
— (2H, cy — (EB) cy 4 2( 48) cy 
+ <A> (2) <M), (4.292) 


Cy = $HLHD ~ 36H) SHB — CHAM) <H,) + <A) <H* 


Next, we substitute (4.146) and (4.147) into (4.292). Using (1.47), 
we easily see that 


Cs SK[H, H,,H,) = ; f 7 f° K[F, Fy Fe] dr do. 
Similarly, the expression for C, ts found to be 
C= ; K ae H,, H,| + e f ae cd (e’) f . F(t”) dt”) 
eit) at’) ce F(t’) a’) 
— = dhe ar & (t’) f. F(t’’) dt") 
+(5 me) AI. 3s al (t’ ) a’) ( is F(t’) dt”). 
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We now divide the region of integration t' < t, t’ <t of the 
third and last terms into two parts, i.e., the region t’ > t’’ and 
the region t’ < t’”. Then we use the identity 


<ént> — <€&) <nd> — Em <O 4+ © Cp <O> = KLE, 0, f] + CDKIE, CJ . 
This leads to the formula 


C= 3K [Fa Hath] + ff] oe Feed + | 


~ JM Ga ae) — Ge) Ge B|eroaen 


To obtain the final result, we need only make the change of 
variables co —- +, 7 > o in the last integral. An expression for C, 
is found by the same kind of argument. The derivation of (4.179) 
is now straightforward. x 


on A \dodr 


CHAPTER 5 


Nonstationary Random Processes 


In radio engineering, one encounters many random processes 
other than the stationary processes considered in Chap. 2. In fact, 
as the subject of statistical radio engineering evolves both expert- 
méntally and theoretically, one has to deal increasingly often with 
the extensive class of nonstationary processes. In this chapter, we 
shall consider only a few very important types of nonstationary 
orocesses. 


1. Processes with Slow Nonstationary Changes 


This kind of nonstationary process is characterized by the fact 
that it behaves almost like a stationary process for long time inter- 
vals, which, however, do not too greatly exceed the correlation time. 
Although the mean value m,(t) and the correlation functions 
kot, t + 7), R(t, t + 73,t + 7),.... depend on the absolute time 
¢ (and not just on the time differences +, 7,, 7, ...), they change only 
slightly during times of the order of the correlation time r,,,. As 
applied to the mean m, and the correlation function &,, this 
condition can be expressed analytically as follows: 


om ok 
on Teor K My , ar Toor Ke + (5.1) 


For such a process, we can define a spectral density 


Sléio,t] =2[_m(t,t+ 2) er de, » (5.2) 


131 
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which varies slowly with the time ¢. In the integrand of (5.2), we 
can replace m,(t,t-+ 7) by m,(t—+7,t) or by the symmetric 
expression m,(t — $7,t-+ $7). The spectral density (5.2) for a 
process with slow nonstationary changes plays the same role as the 
ordinary spectral density for a stationary process. 

The actual processes encountered in radio engineering always 
undergo slow nonstationary changes due to lack of stability of 
parameters of the equipment. These changes are unimportant if 
the time &,/(0k,/8t) which it takes for the nonstationarity to 
manifest itself is much greater than the relevant time constants of 
the given problem. If this is not the case, one has to take the non- 
stationarity into account. 


2. Switched-On Processes 


In electrical equipment, noise processes do not become 
“established” at once; in fact, the influence of the initial conditions 
is “felt” for some time after the equipment its switched on. As time 
goes by, the influence of ‘the initial conditions “‘wears off,” and the 
“switched-on” process asymptotically approaches a stationary 
process. 

As an example, consider a stationary normal process &(t), with 
mean value m and correlation function o*R(z). If the noise process 
é(t) has a known deterministic value £(0) = a at the time ¢ = 0, 
then, according to formula (3.26), the process is subsequently 
described by the conditional mean value 


m,(t) = m + (a — m)R(t) 
and the conditional correlation function 
k(t, t + 7) = o%[ Rizr) — R(t) R(t + 7)). 


These quantities depend on t, and hence the process &(t) ts non- 
stationary. However, after a time considerably greater than the 
correlation time has elapsed (1.e., when t > Teor t + 7 > Teor), 
the functions R(t), R(t + 7) approach zero, and m,(t), R(t, t + 7) 
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4pproach the quantities m, o?R(r) corresponding to a stationary 
process, 

In many cases, a noise process in a piece of electrical equipment 
is described by a differential equation, e.g., a first-order differential 
equation 


é = f(é) + Mt), (5.3) 


‘nvolving an external random perturbation {(t) which is stationary. 
The function f is usually such that as time increases, a stationary 
orocess is established in the system. However, in the beginning, a 
nonstationary transient process takes place, which can be studied by 
using the methods given in Chap. 4, Sec. 4. 


4 


3. Processes with Stationary Increments 


. When the function /(é) in equation (5.3) takes certain forms, the 
noise process £(t) never “‘turns into” a stationary process, no matter 
how much time t has elapsed since the initial time. In particular, 
this is the case for the equation 


£= C(t), (5.4) 


which describes a process with stationary increments. Because of the 
stationarity of the function {(t), the increment 


tt T 
(ty + T) — (te) =" Ue)at 


during an interval of length T has the same statistical properties, 
regardless of which time t) is chosen. (In what follows, for brevity 
we set tg = 0 and &(t») = 0.) If we subtract from the integral 


att) = [Mey ae (5.5) 


its own mean value, and then square and average the result, we 
obtain 


Dé(t) = | | k(t, — t)dt,dt, = 2 | (t —r)k(r)dr. (5.6) 


134 GENERAL THEORY OF RANDOM PROCESSES [cu. 5 


Differentiating (5.6) with respect to t, we obtain 

dad t 

5; Dat) = 2 | Bee) dr (5.7) 
Sometimes it is convenient to write (5.7) in the form 


< Dé(t) z 2. kr) dr ~ 2 i kr) dr 
or 


GD) = FS —<Es0]- 2" kee) ar. (5.8) 


To determine the variance Dé(t) as a function of time by using 
(5.7) or (5.8), we have to integrate both sides of the equation with 
respect to t, using the initial condition 


Dé&(0) = 0. (5.9) 


Once we have found Dé(t), we can easily calculate the correlation 
function &,(t, t,). In fact, if we substitute 


a= &(t,) — <&(t)), bb = &(t,) — C&(t2)) 
into the identity 
ab = }[a? + bt — (a — 5)*], 


and then average the result, we obtain 
R(t, tz) = $ {Dé(t,) + Dé(t2) — D[&(t.) — é(¢2)}} - 
However, because of the stationarity of the increments, we have 


D[é(t:) — €(t2)] = D&(| t — tz 1) 


and hence 
ki(t, t.) = 4 {Dé(t,) + Dé(t,) — D&(| t, — tg |)} - (5.10) 


We now consider some examples. 
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Example 1. Let {(t) be a delta-correlated normal process with 
zero mean value. Substituting the correlation function k,(7) = K8(r) 
into (5.6), we obtain 
Dé(t) = Kt. (5.11) 


‘Therefore, the distribution of the increment of & during time t is 


F l ce 

; WS) = oak | ~ OKt 4" 
Because of the delta-function form of the correlation function 
k-(7); increments of the process é(t) during nonoverlapping time 
intervals are independent. ‘Therefore, &(t) is a Markov process with 
transition probability 


=, 1 (é, page: é)? 
PEs: é) tat Vink exp 5 — Sie . (5.12) 


Sometimes this kind of normal process with independent increments 
is called a Wiener process. According to (5.10), the increment (5.5) 
has the correlation function 


K 
Ret, te) = = lt +t, —|t, —t, |]. (5.13) 


Example 2. Suppose now that the stationary process {(t) has 
the spectral density 

S[l; 0] = aw!-#, (5.14) 

where ; 

O<a<l. 


Then, applying the formula 


J 


ie e-*%P—-] duy = (tr)-?I'(p) ’ 
Q 
we find that 


‘ a ioe) : a ‘T a— 
hea) = 9 Re | eheul-da = = cos [(2 — a) 5] P(2 —a)r 3 


(5.15) 
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for + > 0, or : 
Be 
OO) rahe lente 


where we have used the familiar formula 


(5.16) 


I(1 — x) I(x) = 


sin wx 


involving the gamma function. 

When + = 0, the expression (5.16) is not valid, since it can 
easily be seen that k,(r) takes arbitrarily large values near + = 0. 
In fact, including the value + = 0 in (5.16) is like including a 
delta-function term of the form cé(r), where c is infinite; the 
resulting infinite variance D{ = k,(0) is of the same charaeter as 
the infinite variance of delta-correlated noise, and is related to the 
presence of high-frequency spectral components in S[C; w]. 
However, in actual processes there are practically no spectral 
components with frequencies exceeding a certain upper cutoff 
frequency w,. Hence, the variance of an actual process is finite, 
and has the order of magnitude 


2-a 
a We 
As a > 1, the process {(t) turns into delta-correlated noise, while 
for a = 0, formula (5.15) gives 


kG) as ams (7 >0). (5.18) 


Next, we calculate the variance of the integrated process £(t), 
using formula (5.8). Taking into account (5.14) and (5.16), we 
find that 


“nary i I (5.19) 
dt 2 (a— 1) F(a — 1)sin $20 © 
Integrating (5,19) with the condition (5.9), we obtain 
Det) =2 . @<a<l). (5.20) 


2 I(a + 1)sin $na 
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The case a = 0 requires special treatment. According to (5.8) and 
(5.18), we have 

£Dat) = (5.21) 
dt mt! 


as 


The integral of (5.21) is divergent, but formal integration gives 
Dé(t) = “(In t —In0). (5.22) 


The infinite value In 0 is due to the presence of high frequencies. 
If we introduce an upper cutoff frequency w,, then the integration 
of (5.21) should be carried out from 1/w, to t, rather than from 
0 to t, so that formula (5.22) is replaced by 


Dé(t) =" Inw,t (a =0). (5.23) 
From a knowledge of the variance Dé(t), we can find the 


correlation function of the increment of &() during the time 2. 
Using (5.10), we obtain 


ati tt ~ lt ~ te /* 
k,(ty te) = 4 T@ 4 lysin Fre (0<a<1l) (5.24) 
and 
hAt,,t.) = 2in 2% (a = 0) (5.25) 
yew On | ty — te | 


If we now fix the value of the difference t, — t, = 7 and let the 
time t = 3(t, + t,) approach infinity, then, as can be seen from 
the expressions just obtained, the correlation function will not 
approach a finite limit, so that the ‘‘switched-on” process &(t) 
never achieves a stationary regime. However, under the condition 
+ < t, the expressions (5.24) and (5.25) can be simplified somewhat. 
By making Taylor’s series expansions with respect to +, we find 
that 


(e+ 2) 4 (1-3) 204 MD ee. 0<a<]l), 


in(t+5) + in(t 3) =2Ine— 2 +... (a = 0). 


138 GENERAL THEORY OF RANDOM PROCESSES [cH. 5 


Therefore, with an error of order r%(7/t)?-*, we have 


Ripe 2 Pace @ = < (AE EIS 
e(t ptt) serps OO 


a wt? 


k(t — 5, t+5) = In 


(a = 0). (5.27) 


T 


If the integrated process &(t) were stationary, then it would have 
the spectral density 


S[é: 0] = 4 S{ts 0) =, (5.28) 
corresponding to the correlation function 
1 ae ees 
sd aloes (a + 1) sin $70 ea 


(5.29) 
k(s)=C— Zintr (#=0), 
where the constant C is arbitrary. It is true that for an ideal process 
such that the formula (5.28) holds over the whole frequency range 
right down to zero frequency, the constant C has to be infinite in 
order to satisfy the requirement that the correlation function 
k.(r) be positive definite. However, for actual stationary processes, 
(5.28) holds only for frequencies w > w, which are not too small, 
i.e., which exceed a lower cutoff frequency w,, while for smaller 
frequencies w <w, S[£;w] approaches a finite limit of order 
a/w;**. Then, the constant C in (5.29) is finite and depends on 
w,, and (5.29) can be written in the form 


= a apa . 
Rs) = 4 T(« + 1)sin $70 Cees ren 
k,(r) = 5g ln oe (« = 0). 


These considerations show that for observation times t < 1/w, 
a process ¢(t) with the spectral density (5.28) can be regarded as a 
nonstationary process with the correlation function (5.26) or 
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(5.27), while for observation times t > |/wy &(t) should be regarded 
as a stationary process with the correlation function (5.30). The 
difference between these two cases is tantamount to a difference 
in the interpretation of the constant C in (5.29). 


4. Periodic Nonstationary Processes 


According to (2.1), the statistical characteristics of a stationary 
process are invariant under arbitrary time shifts (corresponding to 
replacing t by t + a, where a is arbitrary). On the other hand, the 
statistical characteristics of a periodic nonstationary process are 
only invariant under shifts by a multiple of a certain pertod Ty 
(corresponding to replacing t by t -+- a, where now a = nT, and 
n is an integer). As a result, the probability densities, moment 
functions and correlation functions of a periodic nonstationary 
process £(t) depend on the absolute time, as well as on the time 
differences, but the dependence on the absolute time is periodic. 
If we represent the process &(t) as a trigonometric series, its 
mean value and (second-order) correlation function can be written 
in the form 


m(t) = > merin(t/To) — m, +2 Re > m,errintt| Te) (5.31) 


nA=-D nol 


R(¢ = 5 ee 5) = k,(r) + 2 Re 2, hal rpetrine7 , (5.32) 
where in general the coefficients m, and k,(r) are complex 
[m_, = mx, k_,(7) = Rx(r)). 

If we shift &(t) by an amount a, where 0 < a < To, we obtain a 
random function €,(t) = & (t + a), whose statistical properties are 
different from those of the original process &(t). In fact, in the 
general case, we have 


(a(t)> = m(t + 2) 4 mt), 
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Using the language of radio engineering, we can say that the 
processes &(t) and &,(t) have different phases. In some engineering 
systems, the phase of a periodic process is very important, and 
such systems are said to be coherent. In other systems, which 
are said to be incoherent, the phase of the process is unimportant, 
since the system does not respond to the phase, and instead carries 
out a time average. When dealing with incoherent sytems, we can 
replace a periodic nonstationary process by a stationary process, 
corresponding to a random phase “‘spread.” 

To illustrate this situation, we find the mean value m and the 
correlation function & of a random process ¢(t) = €,(t), whose 
phase a is a random variable which is statistically independent of 
(t) and has the uniform distribution 


l 
= forO<a<T,, 
w(a) = To 
0 otherwise. 
Averaging first with respect to €(t) and then with respect to a, we 
find that 
e i 
m= <mt+a)>= =| m(t + a) da, 
0-90 
oe (5.33) 
M(t, te) = =| mt, + a, tz + a) da. 
0° 0 
Setting 
m,(t, t’) = m(t)m(t') + A(t, t’) 
in (5.33), using the formulas (5.31) and (5.32), and then carrying 
out the indicated integrations, we find that 


m= mM, 


@ 
Mig(ty, 42) = > My em —t/Te + Ret) — te). 
Nm~-@D 
Subtracting #? from m,(t,, tg), we find that the correlation function 
1S 


2ant 


R(r) = 231 m,, |? COs T. + kj(r) . (5.34) 
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To calculate the higher-order moment functions and the prob- 
ability densities, we can carry out a similar phase averaging. In 
particular, the two-dimensional probability density is 


(Es Ea) te — ty) = zs we(f1, fet) +4, tg +a@)da. (5.35) 


As a result of this averaging, the only dependence which remains is 
that involving the time difference 7, so that the process é(t) is in 
fact stationary. 

The above formulas can also be applied to the case where the 
original process &(t) is periodic, but not random. For example, 
suppose that 


&(t) = Asinwoet = ‘ (tof — e-*ot) . (5.36) 
Then 
2 A 
Ty =>, mM, = Fr, My = My = = 0, k, = 0, 


and formula (5.34) gives 
Aa 


Ar) = —7— COS wT. 


Z 


The function (5.36) corresponds to the two-dimensional prob- 
ability density 


w,(é, t,t, t’) = 8(€ — Asinw,t) d(E — Asinwot’'). 
Substituting this expression into (5.35), we find the two-dimensional 


distribution of the stationary process €(t) == A sin (wot + @), where 
the random phase g = wea has a uniform distribution: 


Ws 5.7) = [2x ~/ A? — €2)-*[8(E, — §,cOSwy7 — V A? — c SIN wo7) 
+ 8(€, — &, cos wor + V A? — £7 sin wor] (5.37) 


| Sin woz | 5(€? + £2 — 2£,£, cos wor — A? sin? wor). 


1 
T 


CHAPTER 6 


Systems of Random Points 
and Related Random Functions 


As radio communication, automatic control, telemetry and other 
fields of electrical engineering continue to evolve, extensive use is 
made of pulse signals, where the positions, durations and other 
pulse parameters are often random, with various kinds of corre- 
lation between different pulses. Thus, in order to design radio 
equipment in a rational way, we have to know the statistical 
characteristics of random processes which are due to the super- 
position of random pulses. In this chapter, we shall devote special 
attention to the problem of calculating the spectral density of pulse 
sequences. The theory of random points presented here can be 
applied not only to pulse signals, but also to other cases where a 
random process can be decomposed into discrete elements or 
events, e.g., in the statistical theory of many-particle interactions.! 


1. Methods for Describing Systems of Random Points 


In many cases, one has to deal with a set of points which have 
random positions in a given space, in particular, on the time axis. 
Such points will simply be called random points. The times at which 


1 For an application of the theory of random points to a problem of radar 
detection, see R. L. Stratonovich, The conditional distribution of correlated 
random points and the use of correlations for optimum extraction of a pulse 
signal from noise, Izv. Akad. Nauk. SSSR, Otd. Tekh. Nauk, Energ. 1 
Avtomat., no. 2, 148 (1961). 
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various random events terminate, the points of intersection of a 
random function with a given curve, the centers of gravity of 
particles in statistical physics, etc., are all examples of random 
points. Of course, by specifying the coordinates of the random 
points, we might convert this problem into one pertaining to a 
family of random variables; however, this way of doing things 
is inconvenient, since the number of random points is often a 
random variable itself, and when this is the case, concepts involving 
space and time relationships (e.g., the concept of stationarity) 
lose their intuitive meaning. 

In the special case where there are no correlations between the 
positions of the random points, the distribution of the points is 
described by the Poisson law. However, the theory presented in this 
chapter includes all possible kinds of correlation between the 
random points, and hence the formulas that we derive are valid in 
the general case. In what follows, we consider the case of identical 
random points in a one-dimensional space, which can be interpreted 
as the time axis, but all our concepts and formulas can be gene- 
ralized to the case of systems of points in a multidimensional 
space and to the case of points of several kinds. 

A system of random points is completely characterized by the 
sequence of distribution functions 


Flt), falta te), Falta» ter tg) +s (6.1) 


where 2), ¢,, ¢3,... lie in the domain of definition of the system of 
random points (ie., the set of possible positions of the points). 
Here, distribution function /,(¢,, ..., ¢,) gives the probability 


dP = [fs(t1 weep t.) + 0(4)]4, see 4, 
that at least one random point falls in each of the intervals 
t<t<t,44),.,t,<¢<t, 44, (4,<4,...,4,< 4). (6.2) 


It is assumed that the values ¢,,...,¢,do not coincide (t, + ¢, if 
t ~j)and that 4 is so small that the domain's (6.2) do not over- 
lap. For random points characterized by continuous statistical 
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properties, the distribution functions (6.]) are continuous and 
the values which they take when arguments coincide, e.g., when 
t; = t;, are conveniently defined by the corresponding limits, e.g., 
as t; >t, A desirable feature of these distribution functions is 
that they characterize local statistical properties at the positions 
corresponding to their arguments, without reference to statistical 
properties at other points in space. 

We now choose an interval0 < t < Tin the domain of definition 
of the system of random points, and we consider the generating 
functional 


Lrfo(o] = (10 + 01), (6.3) 


where n is the number of random points lying in the interval 
0 << T, and 4, ..., 2, are their position coordinates. The class of 
functions v(z) is restricted by certain requirements which depend 
on the nature of the random points and will not be discussed here. 
Then different systems of random points are characterized by 
different generating functionals, as illustrated by the following 
simple systems. 


Example 1. Suppose we know that a fixed number N of 
random points falls in the interval 0 <? < T,, where T, > T. 
Moreover, let each point take any position in 0 <7 < T, with 
equal probability, and let this position be statistically independent 


of the positions of the other points. Then the position of each point 
is described by the probability density 


wt,) = ze for0 << 7, (<j <N), 
1 at 
and therefore 

«4 l z 

Kl + ut,)> =] + FJ, Ut) at. 


Moreover, since the positions of the poinfs are statistically 
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independent, the average of the product IJ[I + v(2;)] is just 
the product of the averages <1 -+ v(t;)>, so that 


L7{u(t)] = [! +p f wa)’. 


Example 2. If in Example 1, we increase the interval length 
T, and the number of points N in proportion, so that their ratio 
N/T, = B, which equals the average density of points, remains 
constant, then in the limit we have 


Ly{u(t)] = Jim [! at Ef ao al = exp Je fee) at 


This generating functional corresponds to a “Poisson system of 
points”, which will be studied in Sec. 3. Examples of more 
complicated generating functionals will be given in Sec. 4. 


In the case of a system of random points with continuous 
statistical properties, the generating functional is related to the 
distribution functions by the formula 


oo t T 
L7{ut)] = 1 4+ Yard, ws i Sty oy ts) O(t,) ... U(ty) dt, ... dt,. 
- (6.4) 


For other systems of random points, the distribution functions may 
have delta-function singularities at points where arguments 
coincide; then, formula (6.4) can be used to define the distribution 
functions in terms of the generating functional (6.3). 

The generating functional completely characterizes a system of 
random points, and in this respect, it plays a role similar to that of 
the characteristic functional (1.60) in the theory of random 
functions. In particular, if [z’, ¢’’] 1s a subinterval of the interval 
[0, 7], then the generating functional L,. ,.[v] can be obtained by 
setting v(¢) equal to zero outside the interval [z’, t’’], just as in the 
case of the characteristic functional. 
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Together with the distribution functions f,, it is convenient to 
introduce the correlation functions (of the system of random points) 


Silty), 8th, te)s E(t, ta, ty), eee (6.5) 
defined by the formula 


= | 


14+ 2 ; ie Jalan iedp gg) BG ee yy 


(6.6) 


= exp De » Bits an toys, : 


. orl 


which holds for arbitrary z,, z,,... and arbitrary ¢,, ¢,,... in the 
domain of definition of the system of random points. The corre- 
lation functions (6.5) bear the same relation to the distribution 
functions (6.1) as the correlation functions (1.64) bear to the 
moment functions (1.63). Thus, applying the formulas (1.69) to 
f,and g,, we obtain 


Alt) = gilts), 
Solty, te) = Solty, te) + gilts) Bilt) » (6.7) 
Fo(tys tar tg) = Balti» tar ta) + 3{gy(t1) Bolter ta)}s + ilty) 81 (te) B1(es), 


where the symbol {...}, has the meaning explained on p. 19. 
A system of random points is said to be stationary if the distri- 
bution functions describing it depend only on the time differences: 


Fatty tay sony bs) = fg(te — ty, oy ts — ty). 


The most commonly encountered case is that of ergodic systems of 
random points. These are stationary systems characterized by the 
fact that the correlations between the events consisting of points 
falling at various different time instants vanish as the time instants 


? Cf. formulas (1.66) and (1.67). 
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are moved apart. In other words, the correlation functions of an 
ergodic system go to zero, Le., 


Edlty +, ts) >90, (6.8) 


provided that at least one difference ¢, — t, of the arguments is 
increased without limit ( | ¢, — t,; | + ©). This does not apply to the 
distribution functions, which under the same conditions reduce to 
products of lower-order distribution functions (just as in the case 
of ordinary moment functions, in the absence of statistical 
dependence). 

Choosing the points 2), ¢2, ... in (6.6) closer and closer together, 
and suitably adjusting the quantities z,, z,,..., we can transform 


(6.4) into 
Ly{v(t)] = exp Pa (i ses [edt sory Lg) U(Ly) ... Ut) Aty ... dt.) (6.9) 


[cf. the derivation of the characteristic functional (1.68)]. From this 
it is clear that the correlation functions can be defined as functional 
derivatives of the logarithm of the generating functional, i.e., 


8 InLfo(2)] 
Baty, +, %3) = Balls) «- Sult,) peelings: (6.10) 
while on the other hand, 
me ACO) 
Faty, seep ts) ~~ Fatt) .. S(t.) vit=o. (6.1 1) 


Certain quantities and functions characterizing systems of 
random points can be expressed simply in terms of the generating 
function. For example, let x be the (random) number of points 
falling in the interval [0, 7]. Then, the characteristic function of n is 


emu = Lafeu — 1]. (6.12) 
The probability that no points at all fall in [0, 7] is just 
Pin = 0} =L,{—1]. (6.13) 


SEC. 1] SYSTEMS OF RANDOM POINTS 149 


Next, we present without proof? some formulas involving the 
conditional distribution functions p(t; T), po(t, t2; 1), ... Here, the 
function p,(t;, .... t3 T) gives the probability 


dP = [p(t ..., t3 T) + O(4)] 4,... 4, (0.14) 


that just one random point falls in each of the intervals (6.2) and 
that n = s, i.e., precisely s points fal] in the whole interval [0, 7]. 
Unlike the ‘‘unconditional” distribution functions, the conditional 
distribution functions are no longer local statistical characteristics, 
since their values at the times 2, ..., 2, depend on the interval 
[0, 7] in question. In fact, the conditional distribution functions 
are related to the unconditional distribution functions by the 
formulas 

©, (_ 
Paty, +) tes T) = > 


r=0 


IyrT pT 
EU il Hea Mtinsea tls Nal at oldisee (615) 
0 0 


r! 


Dividing (6.15) by s!, the number of permutations of the points 
falling in [0, 7] (the points can be distributed in various ways 
among the elementary intervals [#,, 2; + 4,], ...» [ty t, + 4tel) 
and then integrating with respect to the variables 4, ..., 2%, we 
obtain the probability that precisely s points fall in [0, 7]: 


icp at 
Pin =) = P,=> | = | Pallas oy tas T) dty dt, (6.16) 
The meaning of the normalized conditional probability density 
Atal 4: 
w(t, «+ ty) = Pile Ca eee, (6.17) 
is quite close to that of an ordinary probability density. 


Solving the inverse problem, we can find the distribution 
functions /,, /,, ... from a knowledge of the distribution functions 


® See P. I. Kuznetsov and R. L. Stratonovich, On the mathematical theory 
of correlated random points, Izv. Akad. Nauk SSSR, Ser. Mat., 20, 167 (1956). 


150 GENERAL THEORY OF RANDOM PROCESSES [cH. 6 


(6.17) and the probabilities P,, P,,... that various numbers of 
points fall in [0, 7]. The resulting formulas 


~ (str)! ryt 
Sty see ts) = a Puss J, aks a Were(lys wong le rr) Absa ses dt,,, 
(6.18) 


are the inverses of (6.15). If the number of points in n is not 
random, then (6.18) goes over into the expression 


T oT 
Sti sens t,) = es)! = ay J, a8 ih W,(ty, aaj tn) Gigi ys dt,, for s <n, 
(6.19) 
Sty... ¢;) =O for son. 


2. Random Functions 
Constructed from Systems of Random Points 


Using a given system of random points, we can construct random 
functions in various ways. For example, in many cases we are 
concerned with a random function defined as a sum of the form 


lz) = D/O ts), (6.20) 


where the ¢; are random points, Q(z, ¢’) is some known function, and 
z denotes some set of arguments (which in particular might be the 
time ¢). Among all functions of the form (6.20), a special role is 
played by the random function 


&(t) = Dt — ty), (6.21) 
which will be called the ‘random density function.” The more 


general random function (6.20) can be expressed as a linear 
transformation of the random density function (6.21 ): 


nz) = | Q(z, 7) &e’) de’ (6.22) 


Therefore, to find the statistical characteristics of the random 
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function (6.20), we should first find the corresponding characteristics 
of the random density function. 

Just like any random function, (6.21) is completely described by 
its characteristic functional (1.60). If we take account of the 
properties of the delta function, we can write the characteristic 
functional as 


nt 


@,[u(t)] = (exp f f : u(t) &(¢) at) = (exp {i> (e)}) 


j=l 
or 


8,[u(t)] = II eu) ‘ (6.23) 
j=l 

Here, as in (6.3), 2 is the number of random points falling in the 

interval [0, 7], and 2),...,%, are the coordinates of the points. 

Comparing (6.23) with the formula (6.3) which defines the genera- 

ting functional, we find the formula relating the two functionals: 


O,{u(t)] = Lrfe™ — 1). (6.24) 


Thus, if we know the generating functional, we can determine the 
characteristic functional, and hence the other characteristics of 
&(t). In particular, we can determine the correlation functions of 
&(t), which can be expressed in terms of the correlation functions 
(6.5) of the system of random points. The corresponding formulas 
can be obtained either by functional differentiation with respect to 
u(t), followed by setting u(t?) = 0, or by using the expansions 
(1.68) and (6.9). For example,, substituting (1.68) and (6.9) into 
(6.24), taking the logarithm, and retaining only linear and quadratic 
terms in the u(z,;), we find that 


T 
if Ales) alta) dty sb Sf J tas) aes) alts) dt, dy + » 
T 
= iB Ex(ty) [ev — 1] dey 


l T -v 
+ 2 I, J Balti, to) [efutt) = 1] [etl ts) aon 1] dt, dt, Aon 
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or 
if h(t) u(t,) dt, + el ade te) u(t) u(te) + .. 
é 2 er ’ I 2 
et {2 °T 
ait J. By (ty) uty) dty + > ie Bi(ty) u(t,) at, (6.25) 


i2 pT pT 
+f J salto) ath) hte) dts dtp +. 
Then; equating the linear and quadratic terms separately, we obtain 
A(t) = gi(4), 
Ra{ty, tz) = Balty) S(t) — ta) + Solty te). 


Similarly, writing more terms in (6.25), we can derive formulas for 
the higher-order correlation functions, e.g., 


(6.26) 


Ra(ty, ter tg) = 5(t, — te) 8(t) — ty) 8y(ty) 
+ 3{8(t, — ty) Solty, te)}s + Salta te» ts) 
Realty, ty, tar ty) = (ty — te) (ty — ty) 5(t) — ba) By (41) (6.27) 
+ 4{8(t) — t,) S(t; — ty) Solty, ta)}e 
“+ 3{5(t) — t2) 5(ty — tg) Bi(ta, ta)}s 
+ 6{5(ty — t2) Bo(ta» ta, ta)}e + Ballas tas ta» ta) » 


20 eSOe cee nse 8s Gee sbe one 


Here, as on p. 19, the symbol {...}, denotes the symmetrizing 
operation. 

It is not hard to see that substitution of the expansions (1.66) and 
(6.4) into (6.24) gives the same formulas for the relations between 
the moment functions and the distribution functions: 


muy(ty) = f(t) » 
Mo(ty, tg) = (ty — te) fa(ts) + So(ta, f2) » (6.28) 
Mig(ty, ta, ts) = S(t, — tg) 8(ty — tg) fa(ts) 
+ 3{5(t, — te) falty, ts)}s + Soltis ta» ts)» 


FoF FOSS a tHe cnet Ihe tbo 
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From a knowledge of the correlation (or moment) functions of the 
random density function, we can find the correlation (or moment) 
functions of any other random process of the form (6.20), by repeat- 
edly applying the linear transformation (6.22). It will be observed 
that we are dealing with a completely symmetric situation: On the 
one hand, the relation between the g, and the f, is the same as that 
between the k, and the m,; on the other hand, the relation between 
the g, and the &, is the same as that between the f, and the m,. 
Moreover, there is a deep similarity between the mathematical 
apparatus of the theory of random functions and that of the theory 
of random points; in particular, this leads to the appearance of 
syrmmetric sums of the same kind in both theories. 

As already remarked, the results given here can be generalized to 
the case of a system of random points in a multidimensional space, 
by simply replacing z by the coordinates 2,, ....2, of a random 
point in the space. For example, the multidimensional generaliza- 


tion of (6.26) is 


Ry yp +r Zp) = SilAy «+r Zp) » 
Ra(aqy veep Spo Sp vee BM) = BZ oy Z,) (B_ — 2). (2, — 2) (6.29) 


+ Bol) +) B yy Zp very z)) ; 


3. Poisson Systems of Random Points. Shot Noise 


A system of random points is said to be a Potsson system if 
there are no correlations (of any order) between the points, i.e., if 
the occurrence of one or several points (in certain places) does not 
change the probabilities of occurrence of the other points. In this 
case, all the correlation functions of the system of points vanish 
except g,(t,), i.e., 


Baty te) = 9, La(ty ter ts) = 9O,... (6.30) 


‘According to (6.9), the generating functional then has the form 


Tt 
Lrlo(t)] = exp }fex(t) o(e) det. (6.31) 
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If we expand the exponential in series and compare it with the 
expansion (6.4), we easily see that the distribution functions factor 
into products of one-dimensional functions 


Sal tay oes t.) = Blty) « Er(te). (6.32) 
Using (6.15), (6.16) and (6.32), we find that 


P, = + [J, ate) at)" exp | - J axe) ar| (s =0,1,2,...). (6.33) 


The distribution (6.33) is called a Potsson distribution, and is often 
used to define Poisson systems of points. In the stationary case, the 
average density of the points is constant, and (6.33) becomes 


P, = -_ Yeast, (6.34) 

If we have a large number of independent events which take 
place (in time) with a finite average density, then the occurrence 
times of the events have a Poisson distribution. For example, 
consider the equilibrium current flow in a diode. When a single 
electron is emitted by the cathode and reaches the anode, it 
creates an anode current pulse i(t — t,;), where the position of the 
pulse on the time axis is determined by ¢,, the random time at 
which the electron is emitted. Moreover 


i i(t —t,)dt =e, (6.35) 


where e is the charge of the electron. Dealing similarly with the 
other electrons, we obtain a set of random emission times 


aeons t3_4, ty) tesa eee 


which correspond to a system of random points lying on the time 
axis. We shall assume that different electrons are emitted by 
the cathode independently of each other, so that the emission times 
have a Poisson distribution. In fact, there could only be dependence 
between electrons emitted almost simultaneously by neighboring 
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parts of the cathode, but this dependence can be neglected, if we 
assume that the dimensions of the cathode appreciably exceed the 
possible “‘correlation distance.” 

In terms of the average anode current <J,>, the average number 
of electrons which are emitted per unit time is 


_ Ue 
So 


é 


(6.36) 


and this parameter completely determines the system of random 
points. The anode current 


I(t) = >) i(t — &) (6.37) 
3 


‘s a superposition of separate pulses, and hence is a random 
function of the form (6.20), constructed from a system of random 
points. We now use the formulas of the preceding section to find 
the statistical characteristics of J,(t). 

First we assume that the finite size of the transit time from 
cathode to anode can be neglected. If we assume that this time is 
infinitely small, then each separate pulse can be regarded as a 
delta functron, 1.e., 


i(t —t,) = &(t — t,), 
and then the current (6.37) has the form 


I(t) = e }) 8(t — tj) = e€(t), (6.38) 


where ¢(t) is given by (6.21). To find the correlation function 
of the current, we set g, = 0 in formula (6.26), obtaining 


R7) = k(t,t +7) = g,5(z), (6.39) 
which implies 
K[lq, Lac) = e<1a>5(7) (6.40) 


according to (6.38) and (6.36). 
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The Fourier transform of (6.40) gives the spectral density of the 
“fluctuating component” J, — <J,> of the anode current: 


SU, — <Ia3 0] = eS[E— 30] =2eKI.>. (64) 


According to (6.41), which is known as Schottky’s formula, the 
spectral density of the anode current fluctuations is proportional 
to the charge of the electron. This is natural, since the fluctuations 
are caused by the fact that electrical charge is discrete (hence the 
term “‘shot nojse’’). 

Taking account of (6.27) and the fact that the correlations (6.30) 
vanish, we easily see that the higher-order correlation functions 
also have ‘“‘delta-function shape”: 


K[Jq(ty)s «<> La(t,)] = e8g15(t, — te)... 8(t) — t,) 
= e171, &(t, — ty)... S(t, — t,) - 


However, at high frequencies, corresponding to oscillations 
whose periods are comparable to the transit time, we have to take 
account of the duration and shape of the individual pulses, by 
replacing the delta function by some other function G(t — ¢,), 
which is determined by the detailed motion of the electron. If the 
jth pulse is 


(6.42) 


i(t — t,) = eG(t — t,), (6.43) 


then the sum of all the pulses is 
I,(t) =e | G(t — 2’) et’) at”. (6.44) 


This, together with formula (6.39) for the correlation function, 
implies that 


K[la, la] = & [ Ge - 9 Gt +7 aids 
(6.45) 
Keeps J G(r + 0) Go) do. 


Using (6.44), we can easily find the relation between the spectral 
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density of the current fluctuations and the spectral density of the 
random density function; the formula relating them is just 


S[Ig — (Ia; wo] = & | Fiw) |? STE — (£5 4], (6.46) 
where 
Fliw) = | e-*G(r) dr. (6.47) 


Moreover, it follows from (6.39) that 


S[E — <6); w] = 2g, (6.48) 
and hence 
Slo — <fa>3 w] = 2e(la> | F(tw) |? . (6.49) 


This formula differs from (6.41) in that it contains the extra factor 
| F(tw) |?, which might be called the ‘frequency depression 
factor,” for a reason that will soon be apparent. It is easily seen 


from (6.35) and (6.43) that 
F() = | G(r) dr = 1. (6.50) 


Therefore, the formulas (6.41) and (6.48) coincide at zero fre- 
quency. At other frequencies, | F(tw)| <1 and (6.49) gives a 
smaller value than (6.41) for the spectral density of the shot noise. 
If the electrons undergo uniform acceleration in the cathode- 
anode space, which is possible if the electrodes are planar and the 
space-charge density is small, then a simple calculation’ shows that 


2 : 
F(iw) = ag [1 — (1 + wT) e#7) , 
ery. 6.51) 
| F(éw) |? = ory fl + 5 (wT) —coswT — wT sinwT), 


where T is the transit time. 


4 We use the fact that if the initial velocity of the electron can be neglected, 
then T = dV/2m/eV and i(t) = (e/d)? (V/m) t = 2et/T*, where d is the 
spacing and V the voltage difference between the electrodes, and m is the 
mass of the electron. 
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x So far, we have assumed that all the electrons give rise to 
pulses of the same form. However, taking into account the effect 
of the longitudinal component v,; of the initial velocity of the jth 
electron, we should write the jth pulse as 


i(t —t,) = eG(t — 4, v), (6.52) 


where the pulse shape now depends on w,. Adding up the pulses and 
introducing the two-dimensional random density function 


&(t, x) = > S(t — 4) 8(v — a), (6.53) 
3 


we find that 


I(t) =e | i G(t — t', v) €(t', v) dt’ dv. (6.54) 


Here, G(r, v) is a completely known function of two variables, 
determined by the detailed motion of the electrons. Thus, we 
are now dealing with a random system of points in the (#, v) plane. 
Assuming as before that the ‘initial data” of the different electrons 
are independent, we can again regard the points as forming a 
Poisson system. According to (6.29), in the two-dimensional case 
we have 


K[é(t, v), &(t +7, v’)] = er(t, v) 5(7) 8(v — v’) (6.55) 
instead of (6.39), and 


ee ef f G(t — t’, v) g,(t’, v) dt’ dv =e | exe) dv, (6.56) 
since the density g,(t, v) = g,(v) is independent of the time and 
G(r, v) dr = 1 


for any v. Letting w(v) be the normalized probability density of the 
initial velocity, we can write g,(v) in the form 


gx(v) = S22 w(v). (6.57) 
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Then it follows from (6.54), (6.55) and (6.57) that 


KU, Lex] = e J | G(t — t’, v) G(t +7 — t’, v) g,(v) at’ dv 
(6.58) 
= eI, J J G(o, v) G(z + 90, v) w(v) do dv. x 


4. Systems of Random Points Defined by the First Two Moments 


In the theory of random functions, one sometimes considers 
only second-order correlations, i.e., only the first two moments of 
the random function. Moreover, the formulas (6.26) and their 
nverses 


Bilt) = <£>, 
Ba(tr te) = <E(y) &(t2)> — <E(ty)> <E(te)> — <E(t1)> 84, — te) 
show that the first two moments of a random function like (6.21) or 
(6.22), which is constructed from random points, are uniquely 
related to the first two distribution functtons of the system of 
points. Therefore, in a theory involving only the first two moments, 
we can regard the functions g,, g, or fi, f, aS known. In general, 
however, the first two distribution functions do not completely 
characterize the system of random points, and hence, to describe a 
system of random points more completely, it may be necessary to 
consider higher-order moments. Nevertheless, in some special 
cases this ts not necessary, and the system of random points, and 
also random functions constructed from it, are completely deter- 
mined by the first two moments. We now consider two such cases. 


(6.59) 


4.1. Systems of points correlated in pairs. Given a system 
of random points, we say that the points are correlated in pairs 
if all the correlation functions except g, and g. vanish, i.e., if 


&3 = &s4 = + =0Q. 
‘Then the generating functional (6.9) has the form 


T | T pT : , ; 
Erlett)] = exp} f ai(ty vey dt +5 ff edltst'y v(t) oft’) dtat’| . (6.60) 
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4.2. Systems of nonapproaching points. Suppose that all the 
correlation functions can be expressed in terms of two functions 
f(t) and Riz, t’) by the formulas 


Enltiy seep th) = G=l)*? (n = I Attn oo fi(tn) {R(te, ty) ove R(ty, t)}e 
(6.61) 
where, as before, {...}, denotes the symmetrizing operation. Then, 


according to (6.9), the generating functional associated with these 
correlation functions is 


i In[] + SO R(t, t’) Alt’) oft’) a” 


fT R(t, 8°) f(t’) v(t") at" aed 


Lrlu(t)] = exp 


(6.62) 
Here we have used the formula 
S(t =J1In(J4J), 
Sul s 


in carrying out the summation. 
The meaning of the function R(?, 2’) ts revealed by setting 
s = 2 in (6.61); this gives 


BAty, te) = —filts) filte) R(t, te) 
or 


Falta, te) = fa(ty) Alt.) [1 ~ R(t, t2)] . (6.63) 


It is appropriate to call R(z,t') the correlation coefficient, since 
(6.63) shows that the distribution function f, vanishes when 
R = 1. Moreover, when R w |, (6.62) becomes 


T 
Lrlo)] ~ 1 + { _ alt) (8) at 


which, according to (6.4), means that all the distribution functions 
vanish, except the first. If 7,,, characterizes how fast the correlation 
falls off, so that R(t, t’) = | for small time separations ¢ ~ t' < Teor, 
then it is clear that our random points cannot lie near each other, 
i.e., the intervals between random points cannot be much less than 
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Teor. Therefore, the random points with correlation functions 
(6.61) are called a system of nonapproaching points. In particular, 
using the expression (6.62), we can find the probability that no 
points at all will fall in the interval [0, 7]. According to (6.13), 
we have 


T Serer, 
Pin = 0} = exp if. Holl — Jo KEMAH | dt}. (6.64) 
0 JF Ree, t’) fe’) at’ 

1 In many problems, in order to obtain concrete results, it is 
expedient to substitute systems of nonapproaching points, of the 
type just described, for the more complicated systems of random 
points actually encountered. For example, formulas from the 
theory of systems of nonapproaching points will be used in Vol. II, 
Chap. 3, Sec. 2, to calculate the distribution of the durations of 
up-crossings (peaks) of fluctuation noise. 


; 5. Spectral Densities of Various Pulse Sequences 
j 

In this section, we shall calculate the spectral densities of various 
kinds of pulse sequences. We begin by considering a sequence of 
pulses of “‘standard form,” “described by the function G(z). In 
general, the positions of the different pulses along the time axis are 
random, so that the pulse sequence has the form 


nt) = 2, Ge — 4), (6.65) 


where the ¢, are random points. We can regard (6.65) as a linear 
transformation of the random density function (6.21), i.e., 


nt) = f ‘ Gt — 2’) E(t’) at’, (6.66) 


or in spectral form, 
Ne = Fliw) ba, (6.67) 
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where 


F(tw) = J * g-iwe G(r) dr , 


-oO 


Iw = 2 ss e*" n(t) dt, 
l 


bu = ie 


= - 
fo ecter g(t) de. 
—O 
We shall assume that the pulse sequence (6.65) is a stationary 
random process, which means that the system of random points 
giving the pulse positions is also stationary. Then it follows from 


(6.66) of (6.67) that 
<n> = F(Q) <£), 


(6.68) 
S[n; w] = | Fw) |? STE; w) . 


But according to (2.9), (2.12) and the second of the equations 
(6.26) or (6.28), the spectral density S[€; w] can easily be expressed 
in terms of the functions g, and g,, i.e., 


S[é; wv] = 2g, + 2(w) + 4agi&(w) , 


(6.69) 
SIE — <6>50] = 2) + 41), 
where 
y(w) = f et Blt) dr (6.70) 
Substituting (6.69) into (6.68), we obtain 
Sl w] = 4nF%(0) gi5(w) + 2 | F(t) ? [(w) + gi] » 
(6.71) 


S[y — <m>5 o] = 2] Flew) Le) + &)- 


In the special case where the function G(r) is a rectangular 
pulse of fixed height G, and width 7, (see Figure 1), so that 


G, for O<7T<%, 


OG) = 0 for r< OOF r>y, 


(6.72) 
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we have 
eis — 
Fla) = 22 [1 — e], (6.73) 
and then (6.68) and (6.71) become 


= Go7y21 > 
<n) 07081 (6.74) 


w* 


G? « WT 
Sq — ays o] = $22 sin? 28 fy(w) + 24] 


_ Thus, we see that there is no particular difficulty in finding the 
spectral density of a pulse sequence when we kno w the first two 
functions g, and g,. However, if g, and g, are not known, we have 
to calculate the spectral density of the pulse sequence by a different 
method, and then use (6.69) and (6.70) to find g,, y and g, instead. 
We now consider an example of this type. 

_ Let the random points ?; be such that the distances between 
neighboring points are independent identically distributed random 
variables, with a given probability distribution w({). We choose 
t = (as the time origin, and we renumber the random points with 
positive coordinates ¢; > 0 in such a way that ¢, has the smallest 
coordinate. Thus, we have 


h—th&=%, tg — ty == Coy wee 
or 
ade eas Sie aime ey ee oe OR Oe CR ORG -oe Ae Oe OES CRT OR Perna 
hoes (6.75) 
as in Figure |. 
7} 
O41 To To= const 
Gy 
O t thy t 


Fig. 1. A pulse sequence constructed from the random points (6.75), 
where each pulse has height G, and width 75. 
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To calculate the spectral density of the random function (6.21), 
we use formula (2.27), which requires that we pass to the limit 


«— 0 in the expression 
L| 85 i iw] » 


( 


Lie: 91 = { : e-™ &(t) dt = e-Pe[] 4 em 4 erie 4) (6.76) 


where 


is the Laplace transform of the random function &(t). It follows that 


o a 
. 2 — p-(P+P*)ty DUC Fee tl — DOC toothy) 

| LI& pl? =e 2 ae , (6.77) 
where the asterisk denotes the complex conjugate. Next, we divide 
the terms in this double sum into two groups: The first group 
contains the terms for which j > k, and we write j — k = I; the 
second group contains the remaining terms, for whichj < k, and we 
write k — j = 1. Then (6.77) becomes 


@ 


<| LIé; ?] [>> = e—(ptP*)te 15) CPE PMC Heart lg —PUC ge tent Ope) 
kul 1=0 
(6.78) 


oO ow 
4 > EPEC toe HEP Cpa t ent ese rd |, 
jul lo 


To calculate the mean value of (6.78), we use the independence 
of the different random variables £,, from which it follows that 
every term of the double sum can be written as a product of 
characteristic functions corresponding to the distribution w(Z), 
taken with different arguments. Thus, writing 


lu) = J ett afb) dk = ety (6.79) 


(recall that { can take only positive values), we have 


<e-7t> = O(ip), 


: (6.80) 
Ke?) = Gip*) = O*(ip) - 
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Therefore, the mean value of (6.78) takes the form 


<I LE: pl It) = <ertet7 6) |) D Op + 9%) Op) 
ee (6.81) 
> Sip + i*) O*ip)} . 


f= l=] 


+ 
ah 


Summing the series in (6.81) [which is easily done, since they are 
geometric series], multiplying by 2e, and then setting p = 4 — tw, 
we obtain 


€ 7 2 
($f) 

_ ket) Kets) Ofie) 1 O*[w + (te/2)] 

“T= Ge) (T= Oe FG) T= 8 ot GTN 
yinally, according to (2.27), to find the spectral density we need 
only pass to the limit « > 0. If w 4 0, O(w) # 1, we immediately 
find that 


' S[é; #) = 


(6.82) 


2 1, __ O*(w) 2 1 — | O(w)|? 

| 1 — Ow) 1— 6*@)$}° Ol — O@)F’ 
(6.83) 

dince by (1.22), . 

} | — Hi). 


, For the frequency range lying near the origin w = 0, the 
situation is more complicated, and a separate argument ts needed. In 
this case, we use the first few terms of the expansion 


oor FJ =1 for F)o 400+ F)]. co 
which gives 


l +7 O*[w + (ie/2)] 
“ae +>] ' 1 ee ie[2) 


“® a= iw T (2) ve aril [i =F of (w _ *)] (6.85) 


= re + w aL _ Ofw > $)I- 
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We now use the relation 
Gyre 7 27) as «-+0, (6.86) 


which is easily verified directly by integrating both sides. Then, 
taking account of (6.82), (6.85) and (6.86), we see that the spectral 
density S[é; w] has a delta-function singularity at the origin. Adding 
this singularity to formula (6.83), which is valid for w 4 0, we 


obtain 
ie Ore 


The presence of the delta-function singularity 


475(w) 
«oy? 
in (6.87) is due to the fact that the mean value <€> is nonzero. In 


fact, <€> is just the reciprocal.of the mean distance <> between 
the random points, i.e., 


=<. (6.88) 


If we center the random function £(¢) about its mean value, then, 


according to (2.12) and (6.87), 


1 — | O) IP? 
STE — <>; w) = w Tr — 0) F . (6.89) 
It is easily verified that (6.89) can also be written in the form 
_ 4 1+ Ow) 
Sif — <i oe] = 7 5 Ree eal 8) (6.90) 


From a knowledge of the spectral density of the random function 
é(t), we can immediately calculate the spectral density of the 
pulse sequence (6.65). Thus, from (6.90} and the second of the 
formulas (6.68), we obtain 


Stn — Gadi o] = Zilia) PRET TA. 691) 
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In the special case of identical strictly rectangular pulses of height 
G, and width 7, it follows from (6.73) that 


ene ant. .2 27a 1 — | Aw) |? 
These formulas allow us to determine the Fourier transform y(w) of 
the correlation function g,(r) of the system of random points under 
consideration. Comparing (6.69) and (6.90) and using (6.88), we 
find that 


oe) 


w : 93 

Ke) = RT Bay an 

Then, taking the inverse Fourier transform of (6.70), we find that 
eae 7 tor O(w) _ | o _ B@) _ 

le) = 55 J _ em Re 5 16a =n |. [~Bfay S07 de 


(6.94) 


x Using the formulas just derived, we can find the connection 
between the characteristic function @(w) and the Laplace transform 
-L[k(r); p] of the correlation function k,(r) = A(r). Differentiating 
| both sides of (2.14) with respect to 7, we find that 


dk(7) *| 


go = Ll” eimiws[e— a] dw. (6.95) 


‘Then, we substitute (6.90) into (6.95), obtaining 


dk(r) a | ia e-ier;., 14+ 1+ 6) a, 
dr a! i — Oe) 


re 
“EB = ° TT — A—w) ~~" 


The second term in the right-hand side can be written in the form 


(6.96) 


] ste : + Ow’) 5 ie fms, 2 2 
4n<O J tw = Aw ne (w' = —w), 
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which is just the negative of the first term in the right-hand side, 
except that r has been replaced by —r. It follows from (6.79) that 


(ip) = [ ertate at, 


i.e., A(ip) is the Laplace transform of the function w(). Moreover, 
@(ip) is analytic for Re p > 0, and the integral 


mE) = ge [ett (w) do = 5 fet Lip) dp (p = — iw) 


is just the inverse Laplace transform. 
Now consider the function 


| + O() 
Pp 1 — Op) ’ (6.97) 


which is also analytic for Re p > 0, provided that { is not a 
deterministic quantity.® Clearly, (6.97) is the Laplace transform of 
the function 


ax i. pry i+ 90) 1+ 9) 4, 
2mt J 100 


PP mp ts a) 


which vanishes for r <0. Therefore, the first term in the right- 
hand side of (6.96) vanishes for r < 0 and the second term vanishes 
ifr > 0, so that (6.96) can be written in the form 


1 [4p] = ye 1+ OW 
— 2¢o> 1 — OC) 
for r > 0. It follows from 
Rr) = -[-4 ode = tr “Gar 
that 
Lode); p} = 3 \c [Fs 2] —2 [Fol], (6.98) 


7 If £ = ¢ with probability 1, where c is a constant, then (6.97) will have 
poles on the imaginary axis, i.e., for Re p = 0. 
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and hence (6.97) implies 
1 14 O(p) l 


MK PI = 305 1 Op) — ROP ae 
A similar relation can also be written for the function (6.94): 
Led(r); P] = p< 7 eR hs (6.100) 


<o> 1 — Op) p<)" 


Next, we consider a sequence of pulses which are not identical. 
Specifically, let the sequence consist of strictly rectangular pulses, 
all of unit height, whose lengths vary from pulse to pulse, so that 
both the initial points and the end points of the pulses form systems 
of random points (see Figure 2). Thus, the random points on the 
time axis are now of two kinds. Actually, the problem could be 
reduced to the study of a system of random points of one kind, 
which lie in a plane rather than on a line. However, we shall not 
give the general theory here, but instead confine ourselves to the 
special problem just described. 


n 


t t 


Fig. 2. A sequence of pulses with random positions and random lengths, 


Thus, let the initial points t, and the points t; of the pulses be 
numbered in order of increasing time: 


wo Sb ct<t <<. (6.101) 
Then, the pulse sequence can be written as 


At) = Dy Fuel? — ts)» (6.102) 
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where 
for O<t<r, 


l 
5,(t) = fo for t<0O and t>0. 


Instead of the function (6.102), it is more convenient to consider 
its derivative 


H(t) = &(t) = 2, [s(t — ¢,) — 8(¢ — t)], (6.103) 


which generalizes the random density function (6.21) to the case 
of nonidentical points. If we wish to find the spectral density of the 
stationary pulse sequence (6.102), we have to use the fact that the 
spectral densities of &(t) and (t) are related by the formula 


Sts 0) = 4, S64] (6.104) 


or 
l 
S[n — <n) o] = = STE — <£); o] - (6.105) 
We shall only consider the case where the following two conditions 
are moet: 
‘1, The pulse lengths 
ti—t_,=p, (6.106) 


are independent, identically distributed random variables, with 
characteristic function 


8,(u) = <err) (6.107) 
for any /; 
2. The intervals between pulses 
t, _ ti =o, (6.108) 


are independent, identically distributed random variables, with 
characteristic function 


O,(u) = <e'») (6.109) 
for any j. 
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Just as before, we use formula (2.27) to calculate the spectral 
density S[£; w]. Of all the inittal times t,, let t) have the smallest 
positive value. Then the Laplace transform of the function (6.103) 
can be written in the form 


LE; P] = e7 Pte — e-PH 4 e-Ph — em Pa — (6.110) 


This is the appropriate formula for the case where the origin = Q 
lies in an interval between two pulses. In the opposite case, where 
the origin t = 0 is covered by a pulse, we have to write 


Llé; p] = — g-Pty 4 e-Plo — e-P4 4 o-Ps — (6.1 11) 


There are definite probabilities P, and P, for each of the two 
relations (6.110) and (6.111) to hold; for the moment, we shall only 
consider (6.110). Taking (6.106) and (6.108) into account, we 
can write (6,110) in the form 


L[é; | = e~P*of | — e~?Ps + €7PP1— PO, __ e~ PP1— P91 — Phy + dea) 


= eto \ fexp[—plrr tort. +p;+o,)) (6.112) 


j=0 


— exp [—p(p1 + 01 +... +. 9; + pjai)}} - 


To calculate < | L[é; p]*| *>, we multiply (6.110) or (6.112) by its 
complex conjugate. This introduces the double sum 


> 


3=0 


AA, 


Ms 


z 


where 


— g-pt —pt: 
A, = e7?'s — e- Pn, 


which we then divide into three separate sums, corresponding to 
jx=kj>kandj <k. Thus, we write 


MEAP =D VAM=T4+Di+D. Cy 


j= 
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where 


Y= LAAT, Y= DAA = YY AeA, >>. 


kad l=] 


In more detailed form, we have 


2: = e (PtP ite >, {exp [—P(pr + 9, +... + 3 + 9;)] 
j=0 
— exp [—P(p) + a +... + 95 + p54:)]} 
x {exp [—p*(p) + 9) +... +p; + 9y)] 


— exp [—p*(p, + 4 +... + 9, + piu]} » 
(6.114) 


and also 


> = e- (P+ P*)ty 
+ 


x > 2 {exp [—Pler + -. + Peet + Seat) ~ P*(Or +... + pe + ox] 


kmO bes] 


— exp [—p(p, +... + pest + Fx4t) — P*(01 + + On + Presid] 
— exp [—p(p, +. + onss + Pesiga) — P*(p, + + Px + od] 


+ exp [—p(p, + + Gear + Pastas) — P*(9y +o + ox + Paid] - 
(6.115) 


Averaging the last two expressions, we obtain products of the 
characteristic functions (6.107) and (6.109), since the random 
variables p,; and o; are independent. Thus, (6.114) leads to the 


j 
expression 
(X,) = Ce-(PtP*)te > BX(ip ms ip*) Bip + ip*) 
i=0 
x [1 — ,(i) — 9,(p*) + O,(— + 2P*))]. 


Summing the geometric series in the right-hand side and setting 
p+ p* = «, we obtain 


(S.) = Saag ll + Oxied — a) ~ OF. 
endo (6.116) 
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Similarly, averaging (6.115) leads to the expression 


(¥,) = > Dy [ Otic) Otic) Op) OL(ip) 
— OF Nie) Obie) OM ip) OL(ep) — OF (te) OF(te) OF (HP) O5(eP) 
+ G¥+H(ie) Oc) O(ip) OL(ip)] . 


Factoring out 6k 1(#€)95 K(ie)O} (ep) Ok(p), and summing the geometric 
series in & and /, we obtain 


___ <eey 9 (tP) O(P) 
(2 »= Le 1 — 6,(ie) O,(#e) O,(te) 1 — 8,(¢) O2(zp) 


x [1 - Gx — ee) + eG], 


According to (2.27) and (6.113), the spectral density we are 
looking for can be found from the formula 


5 SLgi a] = lim (D,) + lim (D),) + lim «{D, ) 
Se <X,) One <%,) 


after setting ‘ 


(6.117) 


(6.118) 


If w £0, 8,(w)O@(w) ~ 1, the passage to the limit in (6.118) is 
trivial. In fact, since 


O,(te) = 1 — ep) + F$eXp*® +..., 
Oy{ic) = 1 — eo) + $XKO™ $0, 
we have 


€ ] 
I-66 Dro (6.119) 
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Therefore, it follows from (6.116) and (6.117) that 


‘ ] 
lim (3) = Gyegoy 2 — Ole) — OF] 
2 


= Ore Re [1 — 0,(w)], (6.120} 


0, (w 6,(w) l 
hm me >, )= <p) + <a> + <sy 1 — Stat 2) ree (vw) 6,(»)| : 


A special argument shows that the relations (6.120) remain the 
same in the range w ~ 0, 0,(w)@,(w) 1. Substituting (6.120) into 
(6.118), we obtain the formula 


Lore ot — 2 __ pe Ll = Or(w)] [I = Ou) 
el ye eee) ey OY 
In the above derivation, we used the expressions which are valid 
for the case where the time origin t = 0 lies in an interval between 
two pulses. In the opposite case, where the time origin is covered 
by a pulse, we have to use (6.111) instead of (6.110), making all 
appropriate subsequent changes. Thus, to obtain the final result, 
we have to average two different expressions of the form (6.118), 
where the first comes from (6.110) and occurs with probability P,, 
while the second comes from (6.111) and occurs with probability P,. 
However, it is easily seen that replacing (6.110) by (6.111) does not 
change the expressions obtained when we pass to the limit « +0. 
Therefore, the need for the additional averaging disappears, and 
(6.121) is actually the final result. 
To find the spectral density of the pulse sequence (6.102), we use 
formula (6.104), sacar 


2 [1 — 6(~)] [1 — 9(e)] 
+S : —_* Re LE PMOL PAO 6.122 
nel = Sap RT ee Oey ID 
There remains a certain arbitrariness in the definition of the value of 
S[y; w] for w ~ 0. An analysis of the expression (6.121) reveals the 
following behavior for small frequencies 


les 6,(w)] [1 — 6,(w)] w* <p)>*Do + <c>*Dp 


-=6 (w) Ov) 2s [Kp FPP 


(6.123) 
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Therefore, (6.122) tends to a finite limit as w —> 0. If we take this 
limit as the definition of the function w-*S[£; w] for # = 0, we 
obtain the spectral density of the process y(t) — <n): 


sate ois 1 Re LL = Silo) 1 = 6x(e)] 
Sl — <n) 0] = dw-*{Kp) + <oy]-* Re = BEE ON 


Sly — <1)5 0] = 2 <e>*De_+ <s)"Dp (6.124) 


[<e> + <P 


We can derive a variety of special results from (6.124). For 
example, suppose the pulse length is a fixed number 79, so that 


O.(w) = etre , 6,(w) 6,(w) = O(w) = <errty , (6.125) 


where 

C=ptoa. 
Then, from (6.124) we obtain 
Sly — 3 7 


RB 77x Re [a = 6) (1 = e~ #0) (— 21) e”to/2 sin S| 


8 . ,w7, 1—| 6 


~ aX 2 [1 OP’ ee) 


which agrees with (6.92) for Gp = 1. Moreover, if both the pulse 
lengths and the lengths of the intervals between pulses have the 
same distribution, i.e., if 


6,(w) = 8,(w) = O(w), (6.127) 
then formula (6.124) gives 
sec: Spe OO) ee Ee 
Ska — “Dsl = Tags Re Tole) = GG 11+ Oe) e618) 


Sometimes it is convenient to use formula (6.124) after first writing 
it in the form 

l 

Sly — 3 e] = 


4 ] 1 a 
wt <p> + <9 Re [= G(w) f LS 6,(w) ] 
(6.129) 
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* By the same method as used to derive formula (6.99), we can 
use formula (6.124) to derive the Laplace transform of. the correla- 
tion function k,(7). Writing (6.124) as a sum of two complex 
conjugate expressions, substituting into (6.95), and taking account 
of the fact that one of the resulting integrals vanishes, we find that 


dk,(r) 1 (ig pies 1 [1 — 9,()] [1 — 6)] we 
dr U<p +. 3) Jw tw 1 — 0,(w) Ow) 


for 7 > 0. It follows that 


dk(r) 7 11 [1 — xp) 11 — ,69)) 
Ue el-- Gyan 1 eye 10) 


Then, using (6.98), we obtain 
pi cts ee ee at), 0 a 
Mito #l = ~eray pt T= Oy) Ox) et oy*D 
(6.131) 


CHAPTER 7 


Narrow-Band Random Processes 


1. Equivalence of a Narrow-Band Process to 
a Pair of Slowly Varying Processes 


By a narrow-band process, we mean a stationary random process 
whose realizations are close to being sinusoidal oscillations of some 
fixed frequency wo, for time intervals equal to a large number of 
periods 21/w,; clearly, such a process has mean value zero. Figure 3 
shows the typical appearance of a narrow-band process. 


Fig. 3. A narrow-band process. 


In spectral language, a narrow-band process is described by the 
fact that its spectral density is negligibly small everywhere except 
in a narrow frequency band 
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where 4w < wy. If 4w is the smallest such “bandwidth,” then the 
time constant 


] 
T= qo’ 
which is much larger than the oscillation period 27/w , characterizes 
the time it takes the amplitude and phase of the process y(t) to 
change appreciably, i.e., deviations between the narrow-band 
process y(t) and the ideal sinusoid it approximates begin to 
“accumulate” during a time of order 7}. 

Suppose we are given two stationary random processes y,(t) and 
y(t), with zero mean values, whose rates of change are characterized 
by the time constant 7, and suppose we form the new process 


x(t) = y,(t) COS wot — y(t) siN wot . (7.1) 


Then, the process x(t) will be narrow-band if w.S> 1/T), i.e., if the 
frequency w, is so large that the functions yt) and y,(t) do not 
manage to change appreciably during the course of a period 
27/w,. To obtain the correlation function of x(t), we multiply (7.1) 
by the similar expression corresponding to time ¢ + 7, and then 
average the result, obtaining 


CxXX,> = CYLV1_> COS wot COS w(t + 7) — <y1Vor> COS wot Sin w(t + 7) 


— CYeYix> SIN wot COS wo(t + 7) + C¥eVor> SiN Wot SIN w(t + 7) 
or 


<xxe> = F[C Hie) COS woT — (Wey) Sin wor 
+ Cy2¥ix> Sin wor + <a Her) COS wor] 
+ $ [CW Hie) 008 wo(2t + 7) — CH Vexr) SiN wo(2t + 7) 
—CV2Vi_> SIN w(2t + 7) — CHeVor> COS w(2t + 7)]. 
It is clear from (7.2) that in general the process x(t) is nonstation- 


ary, since <xx,> depends not only on 7, but also on ¢. In fact, 
process x(t) is stationary if and only if 


(7.2) 


Wii = Yew » 
CVW1Ver? = — CW2Vir> > 
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Assuming that these relations hold, we introduce the notation 


CW = (yew = 77), 


(7.3 
CWiYor = s(7), 
where s(—7) = —s(r). Then, (7.2) becomes 
(xx, > == 1(T) COS wot — S(7) Sin wor (7.4) 


Writing (7.4) in the form 
cam) = Hr + is) elmer + Br — is) etme, 

and taking the Fourier transform of this expression, we find that 
according to (2.9), 
[x5 w] = p(w + wo) — o(w + a9) + p(w — wo) + o(w — wp), ~— (7.5) 
where 

p(2) = i) e@tr(7) dr = 2 i 7(7) cos 927 dz, 

—o 0 


: : (7.6) 
o(Q) = —i f e8t5(r) dr = 2 | s(r) sin Qr dr. 


It is easy to see that the first of the functions (7.6) is even, while the 
second is odd: 


p(—2) = (2), o( -) = —a(@). (7.7) 


Since the functions p({2) and o({2) are appreciably different from 
zero only in the range 2 ~ dw = 1/T,, and since wy > du, the 
functions p(w + we) and o(w + we) are small for positive frequen- 
cies w > 0, Neglecting these terms in (7.5), we find that 


S[x; w] ~ p(w — w) + o(w — a) (ww > 0). (7.8) 
The relations (7.7) allow us to solve (7.8) for p(Q) and o(2): 


p(Q) ~ $S[x, w + Q) + $S[% w — Q], 


(7.9) 
o(Q) we $ S[x, wy) + 2) — £S[x, w, — Q]. 
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Thus, the functions p(2) and o(2) uniquely determine the spectral 
density S[x;w], and similarly the correlation functions r(7) and 
s(r) uniquely determine the correlation function <xx,>, and con- 
versely. 

Instead of two real processes y,(t) and y(t), we can consider one 
complex slowly varying process 


2(t) = y(t) + tya(t), (7.10) 


which has the correlation function 


<22F> = Cyd + <a> — Kaw + KMD 


7.11) 
= 2[r(7) — i5(7)] 
and the spectral density 
S[z;Q] = 4[p(Q) + o(2)). (7.12) 


Comparing (7.12) and (7.8), we see that the spectral densities of the 
processes x(t) and 2(t) are related by the formula 


S[x; wv] ~ 4 S[z; v0 — wo]. (7.13) 


If we multiply 2(t) by the harmonic time factor e*®e’, we obtain a 
rapidly varying process 
A(t) = 2(t) evel, (7.14) 


which has x(t) as its real part: 
x(t) = Re X(t). (7.15) 
A comparison of (7.4) and (7.11) shows at once that 
Xxx, = % Re [<22*) eter] , (7.16) 


which is equivalent to the following formula relating the correlation 
functions of the real and complex narrow-band processes: 


<xx,> = $Re <2*). 
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One can also consider the imaginary part of Z(t), i.e., the function 
x(t) = Im 2(t) = y,(t) sin wot + ya(t) Cos wot , (7.17) 


which is called the conjugate process of x(t) [not to be confused with 
the complex conjugate]. It is easily verified that 


<8%_) = 1(7) COS wor — 5(7) SiN wor , (7.18) 
<x,) = 1(t) Sin wer + 5(7) COS wor , 


while the spectral density of the conjugate process and the cross- 
spectral density of the original process and its conjugate are given by 
sl ieee . (7.19) 
S[x, #; 0] = —ip(w + wy) + iolw + a9) + ip(w — w9) + io(w — 9). 


Because of the narrow-band condition (wy S> 4w), we can write the 
last formula as 


das ip(w — uw) + io(w — 9) & iS[x; w] forw >0, 
LEU a al alas i) S|, SO) tor ene-O 
(7.20) 


The formulas (7.1) and (7.17) establish a one-to-one corre- 
spondence between the two rapidly varying processes x(t), x(t) and 
the two slowly varying processes y,(t), y(t). In fact, solving (7.1) 
and (7.17) for y,(t) and y(t), we find that 


y(t) = x(t) on + £(f) SIN wot , 7.21) 
ot) = —x(t) sin wot + X(£) COS wot . 


As is easily seen from (7.3) and (7.18), the processes x(t) and <(t) 
satisfy the same relations as the processes y,(t) and y,(t): 


(xx,) = (RH) 
(x8) = (XE) = CHE). 


It can also be shown that the formulas (7.22) are necessary con- 
ditions for the slowly varying processes (7.21) to be stationary, 
provided that x(t) and £(t) are stationary. 


(7.22) 
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We have just studied the narrow-band process (7.1) constructed 
from two given slowly varying processes y,(t) and y.(t). Next, we 
consider the opposite problem of finding two slowly varying 
processes y,(t) and y(t) corresponding to a given narrow-band 
process. According to (7.21), the definition of y,(#) and y,(t) 
depends on the definition of the conjugate process *(t). Suppose 
x(t) is given by a stationary (i.e., time-invariant) linear trans- 
formation 


x(t) = {. G(t — t’) x(t’) dt’, (7.23) 


applied to the original narrow-band process x(t), In spectral form, 
we can write (7.23) as 


ky == Fi (iw)x,, ; (7.24) 
where 
l ms —twet 
rwe= se |e x(t) dt , 
| ee (eee 
ty = a e—*tx(t) dt ; 
fo = Fe [__ eviwexte) 


Fie) = f ie eG) dr. 


Then, obviously we have 
S[z, w] = | F(iw) [2 S[x; @] , 


(7.25) 
S[x, ¥; 0] = F*(iw) S[x; w]. 
Comparing these formulas with (7.19) and (7.20), we see that F(iw) 
has to satisfy the relations 


F(iw) = ae ae ey es (7.26) 
These relations guarantee that (7.20) and (7.22) hold, which in 
turn, together with (7.21), allow us to prove (7.4), (7.18) and the 


other results given above. 
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Formula (7.26) describes the behavior of the function F(iw) only 
in the narrow intervals | w + wo | ~ 4w where the spectral density 
S[x; w] is appreciably different from zero. Elsewhere, the choice of 
F(iw) is arbitrary. The important point is that (7.23) must trans- 
form sin wot into —cos wot and cos wot into sin wet. We now show 
two possible specific forms of the transformation (7.23) and of the 
function F(iw) which have particular advantages. 

If we choose 


F(iw) = —isgnw, (7.27) 
where 
Ww 
sgn a = [ol , 


then (7.23) becomes the Hilbert transform 
ay 1p? xt’) 1, 1? xt—7 — x(t +9) 
H)=2[ nat ==], - dr. (7.28) 


t 
In this case, we can write the complex process (7.14) as 


I 
Vn 


a(t) = att) +180) = pe [ng den + Fe [ete xy ds 


= f2 [i etx, dw. 


This way of defining F(id) has the advantage that it can be gen- 
eralized immediately to the case of processes which are not narrow- 
band. 

However, the integral operation (7.28) is not convenient unless 
we are dealing with problems in spectral form. Therefore, we now 
‘indicate another way of defining the conjugate process, which 
involves differentiation instead of an integral operation, i.e., 


(7.29) 


tw 


Ht) = — —4(8), Fiia) = — =. (7.30) 


This way of defining x(t) is convenient for making the generalization 
to nonstationary processes and for studying situations lying outside 
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the context ot correlation theory, i.e., situations where we use the 
apparatus of Markov processes or the theory of higher monients. 
(Only the first two moments are considered in the correlation 
theory.) The advantage of (7.30) is that it can be used in cases 
where x(t) is described by a differential equation. 

It follows from (7.30) that the complex processes (7.10) and 
(7.14) can be written as 


&(t) = x 4% =x 4 = » «zay= (x + a) em tot (7.31) 


Differentiating the second of these equations with respect to t, we 
obtain 
Cz mel + wx), (7.32) 
an expression which is particularly convenient when the process x(t) 
is described by a differential equation of the form 
# + wtx = ot f (x, af). 
Then (7.32) gives 
& = —twy, e-*0' f (x, x, t), (7.33) 
where 
= 5 (acto + 2* e~ fet) : 
; (7.34) 
x= 20 (zetwet ree 2*e-*ot) 7 
2 
To summarize, in this section we have shown that a single 
narrow-band process x(t) is equivalent to two slowly varying 
processes y,(t) and y,(t). Moreover, we have shown how to go from 
x(t) to y,(t), yo(t), and vice versa, by using linear operations. 


2. Narrow-Band Processes Defined by Differential Equations 
In radio engineering, narrow-band processes are formed in 


frequency-selective circuits (containing capacitances, inductances 
and resistances), which are described by differential equations. 
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Moreover, the signal at the output of such a circuit will be narrow- 
band even when the external or internal fluctuations acting on the 
circuit are broad-band. Therefore, we shall consider differential 
equations containing a rapidly varying random process. We begin 
by studying the second-order linear equation 
KX + ek + x = cf(t), (7.35) 

describing the behavior of a resonant circuit under the influence 
of the noise &(t). Here, « is a small parameter which guarantees that 
the narrow-band condition dw < | is satisfied (note that w, = 1). 
In fact, it can easily be seen that the condition dw < | is equivalent 
to the condition e < 1. 

According to (7.21) and (7.30), the slowly varying random 
functions y, and y, are just 

yy = xcost —<xsint, 


(7.36) 


yy, = —xSint —xcost. 
"Differentiating y, and y, with respect to 4, we obtain 


Jo = —(¥ + x) cost, 


or 
Wy = «(% — £)sint, 


Ps = e(& — £) cost, 
where we have substituted from (7.35). Here, « should be expressed 
in terms of y, and y, by using the formulas 
x = y, cost —y,sint, 
—x = y,sint + y, cost, 
which are the inverses of (7.36). As a result, instead of the single 
equation (7.35), we obtain the system of equations 
: € , € ec 
i Weal Ue ef sin t + 5 IY 08 2t — 5 Ya sin 21 : 
(7.37) 
‘ € e«. € 
Je = — 5V2— «£ cos t — 541 sin 2 — 5 Y2 08 2t. 
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The appearance of the small parameter ¢ on the right tn (7.37) 
confirms that the processes y, and y, change slowly. We shall say 
that the equations (7.37) are in standard form, and we shall refer to 
the double-frequency terms 


571 008 2t , 5y2sin 2, 5y1 sin 2t, 5 Y2 cs 2t 


in (7.37) as the oscillatory terms. The effect of these terms during a 
“period” of the processes y, and y, is small and can be neglected in 
the first approximation; thén, the equations (7.37) reduce to 


e € . 
y= —5”— fsint, 
(7.38) 
; € 
Je = — 5 Ia — €& cost. 


If e is sufficiently small, we can investigate the equations (7.38) by 
using the method given in Chap. 4, Secs. 7-9; this corresponds to 
replacing the actual processes y,(t) and y,(t) by Markov processes. 
Using this method, we can replace (7.38) by the equivalent 
“‘simplified’’ equations 


5 € 
MS Clas mee —€&,, 

3 (7.39) 
Ig = — 592 — be, 


where £, and &, are delta-correlated random processes with zero 
mean values and correlation functions 


Ebi = (7) f : Cée.) sin t sin (t +7) dr, 
<f,fo,> = 47) f <EE,> cost cos (t + 7) d7, (7.40) 


¢f,fo) = 5(7) [_ <é£,> sin t cos (¢ + 7) dr. 
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Each of the integrals appearing in (7.40) can be written as a sum 
of two integrals, e.g., 


| ” £6, sin t sin (¢ +2) dr 
<5) <i cost dr— 3 [ <Ef,) 00s (2t +7) dr. 


Here, the second integral in the right-hand side can be neglected in 
the first approximation, since it represents a nonstationary 
oscillatory contribution, which has only a slight effect on the 
Fokker-Planck equation when averaged over a period, The same is 
true of the other two integrals in (7.40), from which the oscillatory 
parts should also be subtracted. As a result, instead of (7.40), we 
have 


<bE1> = (£b0,) = K,d(7), 
Cb, bes> =0 ’ 


(7.41) 


where 


Kes : f . Cis ose die ; Si 1]. (7.42) 


It follows from (7.39) and (7.41) that y,(t) and y,(t) are identical 
exponentially correlated random processes. In Chap. 4, Sec. 10, it 
was shown that the correlation function and spectral density of such 
a process are given by (4.217) and (4.212), respectively. Applying 
these formulas to (7.39), i.e., setting 


p= 7 K=eK,, 
we find that 
{Wi = CVoVew = r(7) = eKge7€! 1/2 , 


Wier = s(r) = 9, (7.43) 


Ky 


p(2) = Q+ (e/4)’ o(2) =0. 
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Moreover, according to (7.4) and (7.8), we have 


<xx,> = eKge~*!#!/2 cos war , 
Ky 
(w — wo)? + (7/4) 


Of course, these formulas could have been obtained directly from 
equation (7.35), The approach used here is convenient since it can 
be generalized to the case where the original equation is nonlinear. 
Such nonlinear equations will be studied in Vol. II, Part 2, which is 
devoted to the statistical theory of nonlinear oscillations, and 
methods will be given there for constructing higher-order approxi- 
mations in the parameter e. In the generalization to the nonlinear 
case, the slowly varying processes y,(t) and y(t) remain Markovian, 
a fact which is very important, since this ensures that effective 
methods for analyzing the processes are available, 

* Everything just said can also be generalized to the case of 
equations of higher order, both linear and nonlinear. To illustrate 
how to go about making this generalization, we consider the 
fourth-order differential equation 


(p>? +ep +144) (p? +ep9 4+ 1—4)x = E(t), (7.45) 


where « and 4 are constants, and p = d/dt. This equation describes 
the behavior of two ‘‘weakly detuned” resonant circuits in series, 
and has the expanded form 


(7.44) 


S[x; wo] = (w > 0). 


(= + 1) +H) + 2H + 4) + e(@ — Ae) = eE(t). 


Generalizing (7.36), we introduce four slowly varying processes: 


yy = xcost — xsint, 


yp, = —x Sint — zcost, 
yy = 2 cost — 2 F* sine, (7.46) 
4x. x +H 


= — —— sint — —cos?, 
54) € € 
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The inverse of the system (7.47) is 
x = y, cost — y, Sint, 
—k = y,sint + y,cost, 


x +x 
€ 


= ys cost — y,sint, (7.47) 


_ xtés 


€ 


= y,sint + y, cost, 


Differentiating (7.46), we obtain the relations 


yy, = —(%F + x) sine, 


I 


Jo = —(*% +X) cos t, 


in = -[(4E - +1)=4*] sine, (7.48) 


i= [fe +1) 4] cee 


It follows from (7.45) and (7.48) that 


i: = —{-2=4* nal! 


+ (1 + 4)x—#—x+ élsint, 
E4e. (7.49) 
jan = —{-2 + (1 + 4%) 2 —#— x + €] cost, 


Of course, we must next express the right-hand sides of (7.49) in 
terms of y,, Yo, y3 and y,, by substituting from (7.47). As a result, 


we obtain the following exact equations in ‘standard form”: 
{ 


J, = —e(yg cost — y, sint)sint, 

Yo = —€(y3 cos t — y, sint) cost, 

¥s = —2(yq sin t + y, cos t) + (1 + 4?) (y, cos t — y, sin t) 
—e(y3 cost — y, sin t) + é]sint, (7.50) 


Va = —e[2(y, sin t + y, cost) + (1 4+ 4*) (y, cos t — yz sin ¢) 
—e(y, cost — y,sint) + é]cost. 
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Then, in the right-hand sides of (7.50), we have to go over to 
sums of sines and cosines at the zero and double frequencies. In the 
first approximation, we can neglect the double-frequency ‘“‘oscil- 
latory terms,” as well as nonoscillatory terms of higher order, 
which are proportional to e*. Thus, in the first approximation, we 
finally obtain the following system of equations: 


; € 
Yo = — 53> 

B (7.51) 
y= —Pa +5 (I + 4%) y, — eésint, 


n= —%—5(1 + A?) y, —efcost, 


The equations (7.51) are of the form (4.185), and for small e, the 
processes 1, V2, V3, Ya can be regarded as Markov processes. This 
is equivalent to replacing the noise terms sin ¢t and £ cos ¢ in 
(7.51) by delta-correlated processes &,(t) and £,(t) with the prop- 
erties (7.40) which go over into (7.41) after discarding oscillatory 
terms. The resulting system of equations separates into two 
completely independent systems, In fact, if we write y, = —yz, 
these two systems 


jr Ms 
€ 2° 3 
. 142 (7.52) 
er —ya— 7 1 — $25 
and 

Yo Ys 

. ogi (7.53) 
é = ¥s 2 ye ey 


are entirely identical. 
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From (7.52) and (7.53), it is easily deduced that each of the 
processes y,(t) and y.(t) has the spectral density 


= Ko _ 
pe) = eT aT ay? A) =9- 
Then, according to entry 5 of Table 1, p. 25, we have 


oh 
2142? 


7) 1 . 
r(r) = enelt1/2 (cos Sr+a sin S17 ) _ (7.54) 
In the case where the equation (7.45) is nonlinear, the equations 
for the slowly varying processes ¥,, 2) V3) Yq Will also be nonlinear. 
However, the apparatus of Markov process theory can still be 


applied to these processes. * 


3. The Amplitude and Phase of a Narrow-Band Process. 
Rayleigh Processes 


The slowly varying processes y,(t) and y,(#) considered above are 
convenient because they are linearly related to the original narrow- 
band process x(t). However, instead of y,(t) and y,(t), it is some- 
times appropriate to consider the amplitude A(t) and the phase ¢(t) 
of the process x(t). These slowly varying processes have an obvious 
intuitive meaning, and are defined by the relations 


A=V¥P+y¥, gp =are tan (7.55) 
1 
with inverses 
: y, = Aco, ye = Asing. (7.56) 


The processes (7.1), (7.17), (7.10) and (7.14) considered previously 
can be expressed in terms of the amplitude and phase as follows: 


x= Acos(wt +p), # = Asin(we +9), (7.57) 


z= Aet? ‘ £ — Aet@ettte ; 
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The amplitude and phase are related to x(t) in a nonlinear fashion. 
Therefore, to calculate their statistical characteristics, we have to 
know the probability distribution of the process x(t). For simplicity, 
we shall only consider the case where x(t) is assumed to be a 
Gaussian process. Then, knowledge of the correlation function 
(7.4) is sufficient to determine all possible distributions not only 
of x(t) itself, but also of the slowly varying processes y,(t) and 
y(t). In fact, y,(t) and y(t) are also Gaussian with zero 
means, since they can be expressed linearly in terms of x(t). 
Once we know the correlation functions <y,V,,.> = (Yey¥e =7(7) 
and the cross-correlation function <¥,y2,> = s(7), we can find the 
various probability distributions of y,(t) and y,(t). 

Since s(r) is odd, the cross correlation between the values of 
y,(t) and y,(t), taken at the same time t, equals zero, and hence 


Wi, Yo) = _ e~ (i twp (20? | (7.58) 


where o? = r(0). Making the change of variables (7.56) and 
integrating over all possible values of the phase from 0 to 27, we 
find that the amplitude has the Rayleigh distribution 


w(A) = 4 e~A¥ Ro? (7.59) 


A random process A(t) which is the amplitude of a narrow-band 
Gaussian process is called a Rayletgh process. 

To find the two-dimensional probability density w(A, A,) of the 
amplitude, we have to use the distribution of the random variables 
Vv» Vor Viz» gx, Which can be found from their correlation matrix 


r(0) 0 r(r) (rz) 
0 r(0) —s(r) 1(7) 

r(r) —s(r) 7(0) O 

s(t) (rt) 0 (0) 
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Then we go from yy, Ye, Y11 Yor to A, Az, Y, Ys, and integrate over 
all values of p and 9,. As is well known,! the result is 


AA, Q_ AA, A? + At 
u(A, A,) = o(1 — 0%) I (Gm <3) exp | 20%1 — 0%) > (7.60) 
where 
9 = SO t=), 


and I,(z) = J,(#z) is the modified Bessel function of the first kind, 
of order zero. 

: In the case of a narrow-band noise process described by a 
differential equation, the behavior of the amplitude and phase is 
also described by differential equations, just as in the case of the 
functions y, and y.. For example, consider the process described 
by the equation (7.35). Differentiating (7.55), we find that 


' ° + 5 ; 7 = . 
( A= “ry aot , oF aaa Vey. (7.61) 


Substituting (7.39) into (7.61) and using (7.56), to the first approxi- 
mation we obtain the equations 


A=— 54 — ef, cosp — efpsing = «F,, 

(7.62) 
g = 3 [é, sin p — &, cosq] = «Fy, 

where £,(t) and €,(t) are Gaussian processes with the properties 

(7.41). 

‘ To write down the Fokker-Planck equation for the amplitude and 

phase, which vary in the way described by (7.62), we apply formula 

(4.194). First, we calculate the coefficients appearing in (4.194). 


1See e.g., W. B. Davenport, Jr. and W. L. Root, An Introduction to 
the Theory of Random Signals and Noise, McGraw-Hill Book Co., Inc., 
N.Y. (1958), p. 162. 
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It is easily seen that 
0 0 
[ Klay Filde=[ <(, 005 @ + £2 sin g) (£1, 008 p + fgrsin g)> dr 


0 0 
=oostp [ <hifid dr +sinty | <akss> dr 


, (7.63) 
= 5K, 
and similarly, 
0 1 
| _KlFa Fad dr = yx Ko. 
0 
(ie K[F,, Fo] dr = 0, 
(7.64) 


[ res F,,| dr = + { - <(é; sin p — £008 g) 


X (£1, sin p — £, cos p)> dr 


] 


= 34 *: 


Thus, according to (4.194), the required Fokker-Planck equation 
has the form 


AAy) = -<sal(- 3 +a K,) o] + > > Kl-aae + ak 
(7.65) 


which corresponds to the following fluctuation equations for the 
amplitude and phase: 


A € e 

= 74 + aq Ko + fi» 
é (7.66) 
2 


prez. 


Here, &, and &, are the same kind of independent random processes 
as in equations (7.39) and (7.62), with the same properties (7.41). 
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Although the term ¢?K,/2A is of order «2, we have retained it in 
(7.66), because the intensity coefficient K, of the fluctuations may 
bé quite large. It is best to assume that K, is of order | /e, and write 


Ky =*. 


Then all the terms in (7.65) are of the first order in ¢, and it Is 
convenient to write the noise terms in (7.66) as +/en, and +/en,. 
As a result, the first of the equations (7.66), which describes the 
behavior of the amplitude, becomes 


A=-5(A-4)+Ven, (7.67) 


where 
: p> = 0, <i> = «O{7) « 
The one-dimensional Fokker-Planck equation corresponding to 
(7.67) is ‘ e 

2s _ K w 

=A) = 54 [(4-4) | + «aa (7.68) 


We already know the form of the stationary solution (4.49) of the 
arbitrary one-dimensional Fokker-Planck equation (4.33). Applying 
(4.49) to (7.68), we find that the stationary amplitude distributio n is 


w(A) = CetAt20+ina 4 e- A? sx (7.69) 


which, of course, is the samte as (7.59), with x = o%. 

Since the changes in amplitude represent a Markov process, the 
expression for the transition probability ~(A,, A) is of great 
interest, since all possible probability distributions, in particular, 
the two-dimensional distribution 


: w(A, A,) = p(A,, A) (A), 


can be expressed in terms of p(A,, A). Using (7.60) and (7.59), we 
obtain the following expression for the transition probability 


HA, A) = re) 
: » (7.70) 
A, ox Lo (£. AA) ex Ay Ate 
, = a — 0%) °° \T— oo?) Pt 208 — 0%) 
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Because of the relations (7.43), which were found earlier for the case 
described by the equation (7.35), the coefficient Q(7) in (7.70) has 
the form 


O(r) = evel l/2, (7.71) 


If O(7) has a different form, A(t) will not be a Markov process, and 
the conditional probability (7.70) will not play such an important 
role as in the case of a Markov process. Substituting (7.71) into 
(7.70), and replacing o? by «, we find that the nonstationary 
amplitude distribution is 


A AA, ACG 
4) = Ges (scsinh aD) exp] © D(T =e) x(e = DY 


(7.72) 


whose time variation is described by (7.68) if the initial distribution 
is a delta function, i.e., if 


w(A) =8(A—A,) for + =0. 


This is one of several examples where a Fokker-Planck equation 
with a nonlinear “average velocity” K, (cf. p. 58) has an exact 
nonstationary solution, in the form of an analytic expression 
involving elementary functions. 

It is sometimes required to find the statistical characteristics 
of a narrow-band random process x(t) or of the other processes 
(7.56) and (7.57), given the statistical characteristics of the processes 
describing the variation of amplitude and phase. In its general form, 
this is a complicated problem, since the transformation from A(t) 
and 9(t) to the processes mentioned is nonlinear. However, the 
problem can be considerably simplified in the case where the 
amplitude and phase processes are statistically independent of 
each other. We now show how to calculate the correlation functions 
of the processes 


a(t) = Ae‘? , at) = A cos (wot + 9) 
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in this case. Using the independence of the amplitude and phase, we 
can write the correlation function 


H 


(zat) = (AA, e999) (7.73) 
as 

<22t> = (AA, <efle-o2)> | 
The average <e'?-*s)» is the value at wu = — 1 of the characteristic 
function 


O,(u) = Cert 9x9) (7.74) 
‘of the phase increment 9, — ¢. Therefore, we have 
<az*) = <AA,) O,— 1). (7.75) 


‘Then, using (7.16) we find that the correlation function of the real 
‘random process 4(t) is 


, (xx,) = $¢AA,) Re [O,(— 1) e-*"] . (7-76) 


When the phase increment has a symmetric distribution about 
;zero, the characteristic function @(u) is real and (7.76) becomes 


<xx,) = $<AA,) O,(— 1) c08 wor. (7.77) 
‘According to (7.4), this means that 
1(r) ae 4<AA,) 6(— 1), s(7) = 0. (7.78) 


As an example, consider the case where the phase changes represent 
a Wiener process (p. 135), i.e., a “pure diffusion process.” Then 
7; — pis a normally distributed random variable, whose mean value 
is zero and whose variance grows linearly with time: 


D(¢g, — ¢) = K,|r|, 
‘In this case, 
@,(u) = exp{— $.K,|7| u*} 


and formula (7.77) becomes 


.<xx,> = $<AA, eX o!*!/? cos wor . (7.79) 
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4. Quasi-Rayleigh Processes 


Suppose we are given a narrow-band Gaussian process &(t) with 
correlation function o?R(r), and suppose we add to &t) a “sine 
wave” E cos (wot + 3), with fixed amplitude E and random initial 
phase 3 which is statistically independent of g(t), Then, the 
resulting process 


x(t) = &(t) + Ecos (wot + 3) (7.80) 


is no longer Gaussian. For (7.80) to be a stationary process, it is 
necessary that the initial phase 3, be ‘‘completely random,”’ i.e., 
that it have the uniform distribution law 


w(Bo) = 5 - (7.81) 


Then, using the statistical independence of &(t) and E cos (wot + #9), 
we can easily find the correlation function and characteristic 
functions of x(£): 


F2 
<xx,)> = o°R(r) + > COS WoT , 
<eit®) = en-au? a Jo(uE) , 


(7.82) 
Cettativ sts) = exp 


a? 2 
— F(t + 2uu R(x) + u)| 
x JAE Vu? + u® + Quy, cos wor) ; 


where J,(2z) is the Bessel function of the first kind, of order zero. 
The narrow-band process x(t) can also be written as an oscillation 
of the form 


x(t) = B(t) 00s [wot + 4(t)] (7.83) 


with slowly varying amplitude B and phase %, which we denote by 
new letters to distinguish them from the amplitude A and phase ¢ of 
the Gaussian process discussed in the previous section. We now find 
the probability density w(B). First, we write the process &(#) as 


& = Y, COs wot — Yq SIN wot , (7.84) 
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where y, and y, are random variables with the normal distribution 
law (7.58). Substituting (7.84) into (7.80), and comparing the 
result with (7.83), we have 
¥, + Ecosd, = Beosy, yo + Esind, = Bsiny, (7.85) 
so that 
yi + yz = B? + E* —2BE cos (p — 9,). (7.86) 
By making the change of variables (7.85), we go from the distri- 


bution 


Wi» Voy %o) = ome exp ei oH (7.87) 
to w(B, ¥, 3,). Bearing in mind that 
dy,dy,d9, = BdBdpdd, , 
and substituting (7.86) into (7.87), we find that 


2(B, p, 9) = exp |— 515 [B* + B® — 2BE cos (y — 9]. 


B 
(na)? 
Then, integrating this expression with respect to ~ and %, we 
obtain the amplitude distribution 


Se} (=F), 


wB) = = ae _ ss (7.88) 


which is a generalization of (7.59). A calculation of the mean value 
(BY = i] ” w(B) BdB 
0 
leads to the following expression: 
7 , E E} E? - / EB 
By = Foe Prl(t + FE) (ge) +e Gel] 89 
The random process B(t) will be called a quast-Rayleigh process. Its 


importance in radio engineering is due to the fact that a received 
radio signal often contains both noise and a useful signal which has 
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the form of a sine wave whose amplitude can be regarded as 
constant, for simplicity. 

There is a different way in which one can arrive at a quasi- 
Rayleigh process. Suppose that in formula (7.1), 9,(t) and y.(t) 
are stationary Gaussian processes whose mean values are not zero. 
Then the process x(t) will be nonstationary. However, stationarity 
can be achieved by making the time origin (i-e., the time to which 
all other times are referred) indeterminate; in fact, it is sufficient 
to introduce a physically imperceptible error in the location of the 
time origin, with order of magnitude equal to the high-frequency 
period 27/w . Then the amplitude of the process (7.1) will be a 
quasi-Rayleigh process with 


E = Viy>* + (y)*. 


The quasi-Rayleigh process B(t) is not a Markoy process, even 
when y,(t) and y,(t) are exponentially correlated random processes. 
Instead, B(t) is a component of the two-dimensional Markov 
process (B(t), ¥(t)) decribed by the equations 


B B @&  E 

ee op tg OX Er 

: 7.90) 
E. é ( 

*=—syinxt+ FZ (x = —%), 


which generalizes (7.66). Here, £, and €, are independent Gaussian 
delta-correlated processes such that 


> =f =0, (ib = (feb = 2 ir). 


However, for simpliticy, in solving certain problems it turns out to 
be convenient to replace the quasi-Rayleigh process B(t) by the 
one-dimensional Markoy process corresponding to the equation 

B Bo ot. E 1,(EB/c?) 


q 2 + aR + F T(eB/e% + ft en) 


which has the same stationary distribution as (7.88). This replace- 
ment is satisfactory both for large and small values of Ejc. 


PART 2 


Nonlinear Transformations 
of Signals and Noise 


CHAPTER 8 


Zero-Memory 
Nonlinear Transformations 


In studying nonlinear transformations of random signals, just as 
in the case of nonlinear oscillations in general, it is difficult to 
develop a universal practical method of solving problems. Instead, 
different problems require the use of different methods, which are 
justified in each case on physical grounds. It is appropriate to 
distinguish two basic cases, depending on the character of the 
nonlinear system in question: 


1, The simplest nonlinear systems are those for which the value 
of the output function (tf) at any instant of time is determined 
only by the value of the input function £(t) at the same instant of 
time, L.e., 

nt) = ssl, (8.1) 
where g(£) is a nonlinear function (see Figure 4a), Such a non- 
linear transformation is said to have zero memory (or to have no 
inertia). A closely related group of nonlinear transformations are 
those such that the input function ¢(t) is subjected to an additional 
transformation by a linear system Sj, and the output function is 
subjected to an additional transformation by a linear system S,, 
where the action of the systems S, and S, is not affected by the 
presence of the nonlinear device (see Figure 4b). Such a trans- 
-ormation can be written in the form 


nt) = Log(Lis(t)] , (8.2) 


where Z, and L, are the linear operators describing the behavior of 
203 
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the systems S, and S,. We know how the correlation functions 
and quasi-moment functions of random signals transform when the 
signals undergo linear transformations. Therefore, the study of 
transformations of the type (8.1) is equivalent to the study of 
transformations of the type (8.2). 


(b) 


Fig. 4. Examples of nonlinear zero-memory transformations. 


2. A more general and complicated class of transformations 
consists of nonlinear transformations with memory (or inertia), in 
particular, transformations described by nonlinear differential 
equations. In Figure 5, we show one of the simplest examples of 
such a transformation, i.e., a system whose behavior is described 
by a first-order nonlinear differential equation. Let &(t) and y(t) be 
the random input and output voltages, respectively, and let D be a 
nonlinear element (a diode) whose current-voltage characteristic is 


[=F(V), 


Fig. 5. The diode detector, an example of a nonlinear transformation 
with memory. 
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where V = £— 7». Assuming that the internal resistance of the 
input-signal generator is zero, and using the obvious relations 


+h =, r=efha=nr, 


we obtain the differential equation 


Hy d= oe -2). (8.3) 
It is difficult to solve this nonlinear equation in the general case 
where the function F is arbitrary, even for a deterministic input £, 
since the solution cannot be expressed in terms of a finite number of 
integrals. Therefore, it is natural that in the case of random 
functions, the solution of (8.3) involves great difficulties. 

We now make a few remarks about the form of the function g(€) 
appearing in (8.1) and (8.2). This function is called the transfer (or 
dynamic) characteristic of the nonlinear device, and is usually 
determined experimentally from the current-voltage characteristics 
of nonlinear elements like vacuum tubes. Then, to facilitate 
subsequent calculations, one makes some suitable approximation to 
g(€). Three approximation methods which are convenient for 
theoretical purposes are to approximate g(&) by 1) a polynomial, 
2) a broken line (the so-called piecewise linear approximation), and 
3) by an exponential, Each of these methods has its advantages and 
disadvantages. It should be kept in mind that the requirement that 
the approximation be accurate and the requirement that the 
resulting analytic expressions be simple are in a certain sense 
antithetical, and as a rule incompatible. 

In the next two chapters, we shall analyze what happens to 
random signals when they are acted upon by nonlinear systems. 
Our problem can be formulated as follows: Given the parameters of 
the system and the statistical characteristics of the input signal &(t), 
find the statistical characteristics of the output signal 7(t).' 


i Cf. P. I. Kuznetsov, R. L. Stratonovich and V. I. Tikhonov, Passage of 
some random functions through linear systems, Avtomat. i Telemekh., 14, 144 
(1953); Passage of random functions through nonlinear systems, ibid., 14, 
375 (1953). 


206 NONLINEAR TRANSFORMATIONS OF SIGNALS AND NOISE [CH. 8 
1. Transformations of Probability Densities 


For nonlinear zero-memory transformations of the form (8.1), 
the basic solution of the problem just formulated is a consequence 
of the following familiar result: Suppose we know the z-dimensional 
probability density w,(é,, ..., £,) of the random variables 


, = E(t,), erty fs = E(t,) , 


and suppose we want to find the probability density w,(7), ---, 7) 
of the new random variables 


m = Bb + En)» 
F (8.4) 


Wn = Bulb wey En) 


where the functions g,,...,g, are piecewise continuous. If the 
inverse of the system (8.4) is 


f) = hid ory Mn) , 
(8.5) 


&, = h(n, sory Mn) ’ 


where the functions /,,...,4, are single-valued, then the pro- 
bability density w is given by the formula 


wm) 2089 "n) = w;{h,(, cree Mn)» vers hm, ore Nn)] | D, | ’ (8.6) 


where D,, is the Jacobian of the transformation from the random 
variables €,, ..., &, to the random variables 7, ..., 7n, Le., 


ah, hy 
Pear 
Deel > ac. (8.7) 
ah,” thy 
on, = Onn 


In cases where the inverse functions /y, ..., #, are not single-valued, 
we have to sum the right hand side of (8.6) over each of 
the subregions involved. For the one-dimensional case, this 
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transformation rule was derived on pp. 9-10. Applying (8.6) to the 
case of transformations of the form (8.6), we obtain 


Wig» M2) = weLh(m), ACye)] | Arr) h’(n2) | (8.8) 


forn = 2, where £ = h(n) is the inverse of the function g(£). 

Difficulties can arise in applying formulas (8.6) and (8.8) to the 
nonlinear zero-memory transformations of practical interest. For 
example, if the transfer characteristic g(£) in (8.1) 1s a polynomial 
of degree higher than three, then it is hard to find the inverse 
function A(y) in general form, Le., it is hard to solve (8.1) for € 
analytically. Thus, in this case, it is appropriate to approximate the 
tunction g(£). If we make a piecewise linear approximation to g(£), 
in many cases the function 4(y) turns out to be discontinuous 
with an infinite derivative h’(y) at certain points. Then, at these 
points, the probability density of »(t) will haye delta-function 
discontinuities. 

We now consider a few special cases. Let 


m = (61) = £1 = Bly, £0)» (8.9) 
where the inverse functions 
f= h(n) = Wp £, = hm, 2) (8.10) 


are single-valued. The Jacobian of this transformation is 


° 


0 


7 _ Ohy 
Pe =| oh “Ohig. | = One? 
On Ong 
and therefore 
oh 
wma me) = eels halrns 10) | 3 | (8.11) 


To obtain the one-dimensional probability density of the random 
variable y., we integrate (8.11) with respect to 7: 


we) = f ean he (me nal | 5" | én - (8.12) 
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Using (8.12), we can derive the following expressions for the 
probability densities of the sum, difference, product and quotient of 
the two random variables £, and ,: 


(Ey + £2) = v(m) = | wef 7 — &1) db. (8.13) 
w(E, ~ Ea) = w(n) = | we(E & — 7) db, (8.14) 
weak) = wd = f me(bur ) (8.15) 
wo(-$E) = en) = frollan nf) | Ba | (8.16) 


If the random variables €, and &, are independent, with probability 
densities p(f,) and 9q(£,), then in these expressions we have to set 


we(£1, £5) = P(E) 9(&a) - (8.17) 


The next two examples illustrate the use of formulas (8.15) and 
(8.16). 

1. Suppose we want to find the probability density of the 
product of two correlated random variables £, and €,, whose joint 
probability density is normal, i.e., 


= 1 1 Fa 2 2RE£, 
dn &) = ae | a a (GE +> 7 )\. 


Then, applying (8.15) with » = &&, we obtain 


a eee ey eee aes 
a= 17030, V 1 — R? oe 9,0,(1 — By | 


(f+ ait) ae (8.18) 


*K —_ 
oft) fy 


i, <P e 20%(1 = R®) 


_ Ry 


] 
7 10,0, V1 — R? se | o,92(1 — R?) 


K (ar oR) 
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where K,(z) = (7#/2)H{(iz) is the modified Bessel function of the 
second kind, of order zero.” 

2. Next, we calculate the probability density of the ratio 
n = &/€, of two normally distributed, correlated random variables 
é, and &,. According to (8.16), we have 


= l 7 Go 1? 2Ry 
m7) = 10,0, V1 — Rez i exp} ~ 21 — R%) (Gs 3 a? 10, ) fi dey 
0)92 V 1 — R2 (8.19) 


= a(o2 — 2Ro,o, + a*?) : 


2. Moment Functions for Polynomial Transfer Characteristics 


Let the transfer characteristic 7 = g(&) of the nonlinear device 
be an analytic function in a certain interval, so that it can be 
expanded in a Taylor’s series 


g(&) = a + a,(€ —c) + a,(& — cc)? +..., (8.20) 
where 


l 
a, = Be) ° 


Then, depending on the required degree of approximation, we 
retain a certain number of terms in (8.22), say the first n + 1 terms: 


g(é) =r4y + a,(€ —c) +... + a,(E — oe)". (8.21) 


Even if this approach is not possible, we can still always find a 
polynomial which approximates the function g(£) with the required 
accuracy for a given input process £(t). In choosing this approxi- 
mation, it is important that the polynomial be a good approximation 
to g(£) on that part of the £-axis corresponding to values of £&(t) 
which have high probability. However, the approximation can be 
quite inaccurate for parts of the transfer characteristic corre- 
sponding to values of £¢(t) which occur with relatively small prob- 
ability. 


® See I. M. Ryshik and I. S, Gradstein, op. ctt., formula (6.447.2), p. 318. 
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Another way of finding the best approximation to g(£) is based on 
the method of least square error and goes as follows: If the 
approximating polynomial is to have n + I terms, we first form the 
difference 


e(f) — > a,( —c)*. 
k= 


Then the unknown coefficients a, are determined from the 
condition that the integral of the square of the error, 1e., of the 
quantity 


[= | [ete +x) — ya] dx, (8.22) 


should take its minimum value. The limits of integration a and 6 are 
chosen in such a way that the value of the random variable € is 
overwhelmingly likely to fall in the interval a+¢<&<b+¢, 
which means that 


b+¢ 
[ w(@de~t. 
a+c 


If the input process ¢(t) is specified by its moments, then without 
using these moments to reconstruct the probability density w(&), 
it is convenient to set 


cam, b=—a=)A\Vm,— mm. (8.23) 


Moreover, if the probability density #(£) does not differ too strongly 
from a normal distribution, we can choose d equal to about 2 or 3. 
The necessary conditions 

al al 


0 


da, wea gg te 


for the expression (8.22) to have a minimum lead to the system of 
equations 
sole (sDn itk+l gg. — : k 
er ares ed a= tex + ode, (8.24) 


j= 


from which the coefficients ap, a,, --., 4, can be determined. 
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Thus, for the rest of this section, we shall assume that instead of 
the general transformation (8.1), we are dealing with a trans- 
formation of the form 


y(t) = ayx(t) +... + ayx"(t) , (8.25) 
where 


y(t) =7(t)— a, x(t) = E(t) —e = (0) — . 


To find the moment functions of y(t), which will be denoted by 
M,, we multiply the expression (8.25) by similar expressions 
corresponding to different instants of time. We then average the 
result, obtaining 


fiy(t) = dgua(t, t) + dgpug(t, ££) + oe + dntin(ty yt), 


Mie(t,, t2) = ipta(ty, ta) + 24 [M9(t1, fe, £2) + palta, £15 £2)] 
(8.26) 


+ Ayal peg(tys bar £25 to) + oalte bay Lr fa] 


H aapg(ty, ty, te fa) +. , 


Here, the quantities 
Penta, s--y tn) = X(t1) --- #(tn)> 


are the moment functions of the process x(t), and are equal to the 
central moments of &(t) whence = <&(t)>. 

It is apparent from the formulas (8.26) that the moment functions 
of the output process y(t) can be expressed as linear combinations 
of the moment functions of the input process €(t). However, the 
formulas for the moment functions of the output process involve 
higher-order moment functions of the input process. This is 
one of the characteristic features of nonlinear transformations as 
compared with linear transformations. We now consider some 
examples. 
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Example 1. Let the nonlinear device have the transfer 
characteristic 


Wt) = BE) = ag) — agX 2), (8.27) 


and let the input &(t) be a stationary Gaussian process with zero 
mean and correlation function 


k(7) = 0? R(r). 


We wish to calculate the mean value and correlation function of 
y(t). It is easy to see that the first and second-order moment 
functions of y(t) are 


CW(t)> = 4(F(t)> — a(t) , 
C(t) (te) = 436 2(t,) E(ta)> + a8 E(t.) A(t) (8.28) 
~ aga{(£7(t,) E%(t2)> + (E(t) E(t.)>] . 


As is well known, the univariate moments of a Gaussian process 
like £(t), with mean zero and variance o*, are given by the formula 


fl 3+5+*-GG—lo for evenj, 
<6) = fo for odd j. a) 
To calculate the bivariate moments of the normal process &(t), we 
use the expansion (3.14), from which it follows that 


Ri(7) 
n° 


CLE) = aitk > Nin (8.30) 
imo 


where 


N5x = | 33 FUR+N( x) dx . 


—-@ 


The set of coefficients V,, forms a matrix of the form 
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| Fn ef | fy fe 


eS TT ee Oy ee ee oo 


————_ Se ee ee ee ed 


i a cf a | me | 


Se CH ee oe 


SN ed ee ed 


0 0 


This matrix has the following properties, which are easily verified: 


a) All elements above the principal diagonal (j < k) vanish; 


b) The only nonzero elements on or below the principal diagonal 
(j > k) are those whose indices j and k have the same parity; 


c) The nonzero elements N;, are obtained by first forming the 


product 
IG-DG-2---G-k+N 


containing k factors, and then multiplying the result by all odd 
numbers from j — k — | to l, eg., 


Neg =6°5-3°-1; 


d) The sign of N;, is negative if k is odd and positive if & is even. 
Applying these rules and (8.30), we find that 


(£8) = of{] 4 2R%(z)], 
(PED = [3 + 12R%7)], (8.31) 
CEE = 09 + 72R%(7) + 24K]. 
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If we use (8.29) and (8.31), then, according to (8.28), 


(I = ayo? — Bayo", 
iain (8.32) 


<yy,) = a@ot(1 + 2R*) — 6a,4,0%(1 4- 4R*) + a20%(9 + 72R® + 24R%). 
In particular, the variance of y is 


Dy = <»*> — (y>® = 2abot — 24a,a,0% + Maio? . (8.33) 


Example 2. Let the nonlinear transformation have the para- 
bolic transfer characteristic 


H(t) = ayx(t) + agr"(t), (8.34) 
and let the input signal x(t) be the sum 
x(t) = s(t) + &(0) (8.35) 
of the “noise” &(t) and the ‘‘useful signal” 
s(t) = Ecos (wot + $0) » (8.36) 


which is a sine wave of fixed amplitude and uniformly distributed 
initial phase. As in the preceding example, we assume that &(t) is a 
stationary Gaussian process, with zero mean and correlation 
function k(r) = o?R(r). 

Substituting (8.35) into (8.34), multiplying the result evaluated at 
time ¢ by the similar expression evaluated at time ¢ + 7, and then 
taking the average, we obtain 


Cy) == ag<s®) + aCe?) , 
CW.> = ACS.) + AKEEY + aR + dances EE g 37) 
+ 2a2<s*> <E*> 4 a2 EE . 


In deriving the relations (8.37), we have used the fact that the 
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statistical independence of the random functions s(t) and &(t) 
implies the relations 


E> = DED, — GED = DED, 
(ss, £8,> = (85, EE, 88> = CB, 


Then, since 


=—-G)=0, ©=é&>=0, 


we have 


sf>=0, G&=0, sf) =90,... 
We have also used the relations 
<s*s,) =0 ’ (£&,> =0 ’ 
from which it follows that 


Cam) = Ca) = 0. 


Since we already know that 
(££ = o0FR(r), — (£7EZ) = off] + 2R%(7)] 


[cf. the first of the formulas (8.31)], then, to evaluate (8.37) we need 
only calculate the moments of the useful signal (8.36). Transforming 
the expression 


(55,> = E*Ccos (wot + Gp) COS (wot + wot + Hp)» 
into the form 
<ss,> = $ E*cos (2wot + wor + 29) + COS wor) , 


we observe that the first term in the right-hand side vanishes when 
averaged with respect to the initial phase go), whereas the second 
term is not random and remains unchanged. Therefore, we have 


(55, = 4 E? cos WoT ; (8.38) 
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and similarly 


C8952) = REX {I + cos (wot + ¥o)] [1 + cos 2(wot + wor + go)]> 


= +E] + $cos 2wo7). Cy) 


Substituting (8.38) and (8.39) into (8.37), we obtain 
<y> = $a,E® + a,o?, 
<yy,> = 4a2E* cos wor + atc®R(r) + 4.a}E*(1 + $008 2wyr) (8.40) 
+ 2a%oE?R(r) cos wr + a20°E? + ao{] + 2R%(z)] , 


from which it follows at once that the correlation function of the 
output signal is 


k,(r) = 4 @E* cos wr + ao? R(r) + 4 aE! cos 2u,7 


(8.41) 
+ 2a23E*R(r) Cos wor + 2ajotR%(r) . 


Example 3. Let the transfer characteristic of the nonlinear 
device be 
y = a(A —c) +a, ~c} (8.42) 
or 
y—a=aA+a,A, (8.43) 


where 

@=ae*—ace, a,=—a,—2agc. 
Here, we assume that the input A(t) is a Rayleigh process, which is 
the envelope of some narrow-band Gaussian process (see Chap. 7, 
Sec. 3), and has the distribution described by (7.59) and (7.60). 
From (8.43), we find in the usual way that 


(y) = a, + aA) + 4,64 , (8.44) 


((y — 4) 0, — @)> = KAA.) + a1a,(AAD + (AA) + aX AAD. 
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To calculate the moments of the random function A(t) which 
appears in (8.44), we use the distributions (7.59) and (7.60). First, 
we introduce new variables 


A A 
a 28? 4 G2? 
in terms of which 
A =o(2z)”, A, = o(2z,)'". (8.45) 


Using (7.59) and (7.60), we see that the one and two-dimensional 
distributions of the random variables z and z, are 


w(z) = e7, (8.46) 
pe ee A (pm e+e 
u(z, 2,) = 1-0? Iy( io Vzz,) exp eye: . (8.47) 
It can be shown? that w(z, z,) has the series expansion 
= 8 S42) Lal 2,) Loe 8.4 
w(z, 2,) = & 7) n(2) L,(2;) (nip? (8.48) 
n=0 
[cf. formula (4.74)], involving the Laguerre polynomials 
Z d” Ty—-Z 
L,(z) = e Ta (z"e-7) , (8.49) 


which are orthogonal with weight e~? on the interval (0, ~). 

It follows from (8.45) and the formula (8.46) for the one- 
dimensional probability density that the univariate moments of A 
are given by the formula 


CAT) = 21 gr(ztl = 2/g* [evar dz = Vol G sft 1) 
: (8.50) 
~e Qh 


3’ The expansion in question is a special case of the Hille-Hardy formula. 
See Bateman Manuscript Project, Higher Transcendental Functions, Vol. II, 
McGraw-Hill Book Co., Inc. (1953), formula (20), p. 189. 
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where we have used the familiar integral representation of the 
gamma function (factorial), To calculate the bivariate moments, 
we use the expansion (8.48) which implies that 


<AtA*y = Qlr4af2qrts( gr /Ags/ — airtay/agrss Dd Arnltsn ae : (8.51) 
n=0 
where h,,, denotes the quantity 
k= | sl¥e-2L (2) dz. (8.52) 
0 


Substituting (8.49) into (8.52) and integrating by parts m times, we 
find that 


[G*)!P 
(gr — n)! 
(8.53) 


hyn = (—1)"5 (5 -_ 1} - 6 —n + 1) fe e~? 2/2 dz = (—1)" 


If r is even, then the quantities A, vanish for » > r/2. Therefore, 
the expansion (8.51) contains only a finite number of terms, 
provided that at least one of the numbers r and s is even. According 
to (8.53), when r = 2 we have 


| a hy = —l, Nog = hy = .. ee 6 (8.54) 


When r = 1, all of the coefficients (8.53) are nonzero: 
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Substituting (8.54) and (8.55) into (8.51), we obtain 
(A*A® = 4o%(1 + O°), 
(AA®) = (APA,) = 09 V2a(I + 502) (8.56) 


1 l l 
(AA) = Fol +508 + GO! + ac O" + 


Then, using (8.44), we arrive at the final result: 


cy) wliaieal steps 
T l 1 l 
Cy — 4) (¥, — 4)> = 5 ato*(I +40 + ao + 566 0° + as 


+ 2aja,o? VIn(1 + 50%) + datet(l + 0%), 
(8.57) 
7 


a(n) = Zaot(O# + LO + Oe ..) 


+ Vie ala,c®Q? + 4a%otQ? . 


3. Moment Functions 
for Piecewise Linear Transfer Characteristics. 
The Direct Method 


The moment functions of random signals subjected to piecewise 
linear transformations can be calculated by two intimately related 
methods, one involving the use of series expansions of the two- 
dimensional probability density (the direct method) and the other 
involving the characteristic function and the evaluation of integrals 
(Rice’s method). In the next two sections, we shall use typical 
examples to illustrate the gist of these two methods, in keeping 
with our policy of deriving quantitative results of practical interest. 


3.1. Gaussian input processes. First, we calculate the mean value 
<> and the correlation function k,(7) of the process 7(t) obtained at 
the output of a nonlinear device with a piecewise linear transfer 
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characteristic g(€), of the kind shown in Figure 6, when the 
input is a stationary Gaussian process &(f) with zero mean and 
correlation function 


R(r) = o° R(r). 
In this case, the one-dimensional probability density is 
—ipreé 
w(é) = -F(°), (8.58) 


where 


1 / , 
= —- [2 
F(x) Wa [. e-¥7/4 dy | 
and the two-dimensional probability density w(f,&,) can be 
conveniently written in the form (3.14). 
Thus, we assume that the function g(&) is continuous and 


piecewise linear, and that its derivative g’(£) is discontinuous at the 
points C,, cz, ..., ¢, (see Figure 6). This means that 


“=> b,6(é —¢,). 8.59 
£'(8) = DoE ~ 4) (8.59) 

Then, the mean value 
(n> = f eymeyae =1{ ey r(®) ag (8.60) 


can be written in the form 


(> = fim | i: e(£) aF() ; (8.61) 
Integrating by parts, we obtain 
[ee ar() =aenr(4)-[" era. 662) 
We now introduce the function 


FOU(x) = [ * F(x) dx = xF(x) + F(x), (8.63) 
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which has the properties 

4 F-14) = F(x) FCW —oo) = 9 

dx ' ° 


Using these properties, and carrying out a second integration by 
parts in (8.62), we obtain 


g ) (8.64) 


ef 


f, 1 
J a6 ae (C) = ate (2) ~ 0 [eo arnn( 
a 

= (és) F(£4)—og'(e)P-4(1) + of | eq ren() ae. 

According to (8.61), the mean value <y> can be obtained by taking 

the limit of (8.64) as £;->o. Since, as £;-» 0, F(&,/o) rapidly 

approaches 1, while F‘-'(&,/o) approaches £,/o [cf. (8.63)], it 
follows that 


o> = dim [el6) — eel tof e"or-n() de, 


&,00 reg 


g(¢) 


Fig. 6. <A piecewise linear transfer characteristic and its derivatives. 
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or 
; . c 
Co) = fim [a@) — gel +o DbeeM(Z), (8.65) 


where we have used (8.59). Moreover, from (3.14) we find that the 
second moment of y(t) can be written as a series of the form 


m9.) = >|, SEL. g(£) ) Fins (é ) ag) = Ro) ; 


whose first term is just <y)>?, according to (8.60). Subtracting this 
first term from <77y,>, we obtain the correlation function 


k,(r) = Dal _ BE) Fen ( fy ae) =e) | (8.66) 


To evaluate each of the integrals in (8.66), we again integrate by 
parts twice. For x = 1, we obtain first 


fear Qys=-feortye, 


and then 
J “a e “() “ = — og'()F(~) + {eer dé. (8.67) 


For n > 1, the integrals can be evaluated even more simply, since 
the corresponding integrated terms vanish at +. Finally, substi- 
tuting (8.59) into (8.67), we find that 


f_sor()2 = —of(@) +0 DbF(*), 


tal 
[a rns (8) & = oD bren( 2) (n> I). 


We must now substitute the expressions just obtained into 
formula (8.66). A variety of special cases are possible, as shown in 
Figure 7. If g(€) consists of a single line segment, so that r = 0, 
b, = 0, then only the first term remains in (8.66). This case 
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corresponds to amplification of the signal | g’ | times, together with 
a possible change in the d-c component of the signal. Next, 
suppose that g(£) consists of two line segments, joined at the point 
& = ¢,, where g’ = a to the left of c, and g’ = B to the right of 
c, (see Figure 7a). Then 


g(~) =8, b, =B—a, 


and we have 


Ho)= fare —Or( 2) Reo +8 — or $ fro yp BO. 


n! 
(8.68) 
g(é) 
> 
Oo 6¢, r 
tang=S 
(b) 
g() 
0 ¢, r 
(c) tang=S (d) 


Fig. 7. Typical piecewise linear transfer characteristics, 
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In particular, if the ‘‘break point’’ is centrally located, i.e., c, = 0, 
then 


RO) 
(8.69) 
pie ee = (+B) R(r) oe  (p—a)? (R arc sin R+*/] —R? — 1). 


ha(r) = (a +B) Rr) + o%f — a? > [Feo] = 


By examining the case 


S(é — cy) for é>q, 


af) = jo or eee (8.70) 


(see Figure 7b) where 
a=Q, p=S, 


we can consider ‘‘peaks’’ of the random function &(t) and we can 
calculate the degree of correlation between them. In this case, we 
have 


k(t) = Sto} Fe(— £h “1) R¢) + Sa b [Fee It Rr). (8.71) 


In Figure 8, we show the results of calculations of the correlation 
coefhicient 

k, (7) 

R,(0) 


based on this formula, when terms up to 2 = 7 are retained in the 
series and when the correlation function of the input process is 
R(r) = e—2" (shown as the dashed curve). Curves corresponding to 
five values of the threshold y = c,/o are plotted. 

As an example of a transfer characteristic with two break points, 
we consider the asymmetric limiter 


R,(7) = 


Sc, for &€>c,, 
n= ef) = {SE for y<&<G, (8.72) 
Se, for <a 


(see Figure 7c), where 


g(~)=0, b=S, b=—S. 
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In this case, the correlation function is 


k,(r) = S?o* > Fee oa — Fen 4) RX) : (8.73) 


o n! 
and, according to (8.65) and (8.63), the mean value is 
aa ul rec) fo (2) — oF { & 
a) = Sa-+ fet) ~ aff) + oF (2) ~ oP (4) 

(8.74) 
To find the variance of the output signal in each of the examples 
just discussed, we can set 7 = 0, R= 1 in formulas (8.68), 
(8.69), (8.71) and (8.73); this gives the variance in the form of an 
infinite series. However, we note that it is simpler to find the 
variance in the ordinary way, by using the one-dimensional prob- 


ability density w(£) to calculate <n?) — <n>?, since this will give 
the result in finite terms. 


Fig. 8. The correlation coefficient R,(r) corresponding to (8.72), 
when R(r) = e-2, 


The method used above, which is based on integration by parts, 
is also effective for calculating higher moments., The method can 
also be generalized to the case of piecewise continuous transfer 
characteristics of more general form, e.g., characteristics with 
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jump discontinuities or characteristics made up of parabolic 
pieces. For example, if the transfer characteristic is 


q for E>q, 


oe i for <q (8.75a) 


(see Figure 7d), then a single integration by parts is enough, while if 


= B(é — c,)? for >a, 
1= 9 for a (8.75b) 


then three integrations by parts are needed. The resulting expres- 
sions for the correlation function are 


k,(7) = 7d a [Fm(2)]" Rr) (8.76a) 
and 
k(t) = apo > [Fo-(— Ay Rr), (8.76b) 
respectively. 


3.2. Rayleigh input processes. We now consider two cases where 
the input signal is a Rayleigh process and the transfer charac- 
teristic is piecewise linear, 


Example 1. Suppose the input process [whose one-dimensional 
probability density is the Rayleigh distribution (7.59)] undergoes 
the transformation 


q for A>A,, 


7=10 for A<A). eam) 
Introducing the process 2(t) = A?(t)/20?, as before, we have 
q for z> 25, (8.78) 


2°10 for z<2, 


where 
_ Ao 


% = 559° 
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Then, using the distributions (8.47) and (8.48) of the process 2(t), 
we find that 


n> = q [ e~* dz = qge-*o, (8.79) 
sa5 2" ((* “egilt 
Com) = FD Capp [f Ealadent dal - (8.80) 


To evaluate the integrals appearing in (8.80), we use the definition 
of the Laguerre polynomials, obtaining 


(0) qn-} 
—Zz a Ro—Z 
J Ea) e-? dz agai (2"e—*) a : 
which can be written as 
| L (2) e-* dz = —2z,L') (z,) e-70, (8.81) 


in terms of the assoctated Laguerre polynomials 
L(x) = artet © (antte-s) (8.82) 
ie dz” 


Therefore, according to (8.79) and (8.80), the output process, which 
is actually a sequence of pulses, has the correlation function 


bile) = ae 8 Gp EP (8.83) 


For reference, we write down the first few of the polynomials (8.82): 


Lz) = 1, LY(z) = 2—2, Lz) = 6 — 62 + 2%, 
(8.84) 
L2\(z) = 24 — 36a 4 128? ~ 24, .. 


Using (8.84) and substituting z) = A$/20? into (8.83), we obtain 
_. 2 Ao 5 atyot A \* Ay , 4b 
halo) = oF gee HHO +(1 — Ze) Ot+ (1 — se tgqa) OP +]. 
(8.85) 
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We could calculate the variance of 7(t) by setting 7 = 0,0 = 1 
in (8.85). However, it is simpler to make direct use of the one- 
dimensional probability density; this leads immediately to the 
formulas 

(oP) = gre, 


(8.86) 
Dy = k,(0) = q?e-o(e% — 1). 


Dividing (8.83) or (8.85) by the variance Dy, we find that the 


correlation coefficient is 


R(x) = 2) — Blo) + BOMr) + BO%) +..], (8.87) 
(0) 


where 
4 
By = fh (ett — 1p, 
A? 
a a (I a at) : 
S A, , Ady? 
B, = (1-3 + Maal) 


In Figure 9, we show the results of calculations based on formula 


fe) i 2 yat 
Fig. 9. The correlation coefficient R,(7) given by (8.87), when Q(z) = ena 
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(8.87), where only the indicated terms of the series have been 
retained, These calculations were made for the case where 


O(r) = e-=" 
(shown as the dashed curve). The quantity 
ities A 


specifies the operating level of the limiter, and we have plotted the 
curves corresponding to the values y = —1,0,1,2 and 3. It 


should be noted that 
a Ne ew 1.25 
o 2 


for a Rayleigh process. 
Example 2. Suppose the input Rayleigh process A(t) under- 
goes the transformation 


y= {54 -4e) for A> Ay, 


0 for A<A,, A888) 


which ‘‘cuts off the envelope peaks’? exceeding the threshold 
A = Ay. Using the univariate Rayleigh distribution (7.59), 
we obtain the mean values 


<n» = So V2n F(—«) ? 
(n> = 2S%o%[e-F/2 — V2 aF(—a)], 


(8.89) 


and the variance 
Dy = S%o[2e-@? — 2/20 aF(—a) — 20F*(—a)] , (8.90) 


where a = Ao/o. 

To find the correlation function k,(r), we use the expansion 
(8.48) of the bivariate distribution. First, we note that the trans- 
formation (8.88) is equivalent to the transformation 


= zl? — 2z,/? for z>2, 
9 = V2S0 |) for z<%, 2}) 
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where, as usual, 


a et? 


After subtracting <7)? from (8.48), we have 


h,(r) = 2S%o? ar ate O"(r), (8.92) 
where 


a | (2t/? — 21/2) L,(2) e-* dz. 
Z9 
Next, we integrate this last expression by parts, obtaining 
h, = — | _ ALD (2) e+ dat, (8.93) 


where we have used the relation 
L,(z) €-? dz = d[2L™,(z) e-*] , 


which is equivalent to (8.81). 
Introducing the parameter p», and then differentiating with 
respect to z, we can write (8.93) in the form 


n= ua 


env dal? | (8.94) 


Tie § 


Then, setting z = A?/2o0? = a?/2, we express the integral in 
(8.94) in terms of the probability integral (see p. 42): 


[is ene dz? — — ! vA en Ha?/2 dq = JER(-aVi 


It follows that 
hy = Vi 5b (— 2) Palen .7) | 
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For computational purposes, it is more convenient to use the 
somewhat different formula 


on _ 
eee sr E @/2) F(x vu)| Lan (8.96) 
which can be verified by using the relations* 
~2L (2) = a (; ) (n= 1) (n= 2). U2}, 
non n prt ee 
— — 2) | 12 
2 (7) = 1) (= 2) =>) oe 
on" 
= Op” (uF), 
where 
l 
F, = —F(—a Vn). 
1 Vu ( #) 
The first few coefficients h,, are 
hy = — ME ER(—a) + a), 
fee V7UF(—a) + oF (x) + o2F(a)] , (8.97) 
hy = — ive [F(—a) + aF’(x) + 202 F(x) + § 08 F’'"(ax)]. 


If we retain only the coefficients (8.97) in our calculations, 
setting all the others equal to zero, then the correlation coefficient 
of the process 7(t) [which is just the random sequence of peaks of 
the envelope A(t) exceeding the threshold Ag] is given by the 
expression 


R,(r) = B (20° +3 4 22" + + Ho") (8.98) 


4Cf. I, M. Ryshik and I. S, Gradstein, op. cit., formula (7.140.1), p. 381. 
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where 
B= e-@2 — WV 2q a F(—a) — FX —c). 


(Cf. formula (8,90).] In Figure 10, we show how R,(r) depends on 


O 35 O.7 105 Var 


Fig. 10. The correlation coeffictent R,{ +) given by (8.98), when Q(r) = ena 


the value of the threshold 


y= “8 1.25 = —1,0,1,2,3, 


in the case where 


Ol) =e". 


It is clear from the figure that the correlation coefficient of the 
peaks can differ appreciably from the function Q(7), shown as the 
dashed curve in Figure 10.5 In fact, R(r) ‘spreads out’’ for small 
thresholds, and then “shrinks,” starting from some value of 
y ~ 0. 


: 5 Q(z) can be regarded as a kind of envelope correlation coefficient, but, 
as shown by (8.56), O(7) # <AAD — <A>?. 
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4. Moment Functions for Exponential Transfer Characteristics. 
Rice’s Method 


4.1. The moment functions of the output 
n(t) = here (8.99) 


of a nonlinear device with an exponential transfer characteristic can 
be expressed in terms of the characteristic functions of the input 
signal ¢(), This fact can be exploited in cases where the charac- 
teristic functions are known, e.g., when &(t) is a Gaussian process. 
By averaging (8.99), we obtain 


<n> = hO,(—1a) ’ 


; (8.100) 
<n = h?@,(—ia, —ta) ’ 
where 
O,(u) = Keil ’ 


(8.101) 
O,{uy, ue) = cermrtttertey 


are the one and two-dimensional characteristic functions. If &(t) 
is a stationary Gaussian process with zero mean value, then as we 
know [cf. (3.1)] 

O,(u) = exp {— $o°u*}, 


(8.102) 
@,(u,,u,) = exp {— $ o7[u? + 2R(r) uu, + u2]}. 


Then, we can use (8.100) to find expressions for <>, (n7,> and 
the output correlation function 


Ri(r) = Areot[ertoRir) 1], (8.103) 


When the transfer characteristic of the nonlinear device is not 
exponential, we can still use characteristic functions to calculate the 
moments of the output process. To do so, we use Rice’s method, the 
basic idea of which is to represent the transfer characteristic of the 
nonlinear device by a contour integral 


a=8) => f _ Fa) et de, (8.104) 
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where L is a suitably chosen contour of integration in the complex 
Q-plane. If the function g(€) vanishes at infinity (€ = +0), we 
can choose (8.104) to be the Fourier transform; then 


FQ) = [eye a8, (8.105) 


and we need only consider real values of 2. If g(Q) vanishes for 
& < 0 (or at least for € less than some fixed number), we can apply 
the theory of the Laplace transform, by regarding (8.104) as the 
inversion integral 


(2) = 5.3) F(P) dp 


ce 


of the transform 
Fp) = J ert e(Q)dé (p= i). 

In this case, if g(é) grows no faster than some power of é, as 
& —» + o, we can choose the contour of integration in (8.104) to be 
the real axis (c = 0), with a downward indentation at the origin. 
(The indentation is to the right inthe complex plane corresponding 
to p = 122.) If g(€) goes to infinity in both directions (1.e., as 
€ —» +o), then it is appropriate to use the theory of the two-sided 
(bilateral) Laplace transform. 

Averaging first (8.104), and then the product of two expressions 
of the form (8.104) evaluated at the times ¢ and ¢ + 7, we can 
express the moments <7) and (77, in terms of the characteristic 
functions of the input process, which are assumed to be known: 


<n> = 52 f FUR) <exp (326}> dO, 
(8.106) 
don.) = gaff FU) PUR.) exp (QE + 2,£)}) 40 aQ, 


For example, if &(t) is a stationary Gaussian process, then, according 
to (8.102), we have 


dame> ~ Gea J J FQ) FUR, exp (— bo(OP+2RO0, +.) dO a0, 
(8.107) 
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If this integral cannot be evaluated explicitly, we substitute the 
expansion 


e-PRQQ, py oe  2nRngangn (8.108) 


na0 


into (8.107), obtaining 
<A 
Gans) = > oR), (8.109) 
nod 
where the coefficients 


Iy = 5 =f F(iQ) e-2°9"/ Q* dQ (8.110) 


are much easier to compute. Then, using the first of the formulas 
(8.106), we subtract (7>? from (8.110), which gives the correlation 
function 


(- 6) 


h? 
k, (7) =>* a" R(r) , (8.111) 


n=l 


Example 1. To illustrate Rice’s method, we find the correlation 
function k,(r) corresponding to the nonlinear transformation 


AVI —&(t) for |é| <1, 
w= ‘5 ae (8.112) 


In this case, we can simply take the Fourier transform (8.105), 
obtaining 


1 
F(iQ) = Af e-ink 4/] — 2 dé, 
-1 
where the integral is readily expressed in terms of a Bessel function,® 
i.e, 


Ria) = 7A LOD, (8.113) 


¢[. M. Ryshik and I. S. Gradstein, op. cit., formula (6.413.3), p. 312. 
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and the inverse transform is just 


A? Si) pice 

oak aaa as aQ . (8.114) 
For Gaussian noise with the characteristic function (8.102), the 

desired correlation function is given by (8.111), where the coeffi- 


cients (8.110) are 


ha = 5A [ Jy(Q)e#. Or a0 (8.115) 


Since ],(2) is odd, the integral in (8.110) vanishes unless 7 is 
even, L.e., 2 = 2m. Introducing the notation 


oy 20 
Log (1/c) = 04 | J (2) e-0°8/2 QA-1 dQ = | Jp (*/o) e727 /2 yA-1 dx c 
0 0 
(8.116) 
we have 
0 for odd zn, 
h, = . (8.117) 
BaF 3 ile) hoki me 


Then, substituting (8.117) into (8.111), we obtain 


k,(r) = A? > apy fhm (Ifo) R°"(r) 


The coefficients g,, can be expressed in terms of the confluent 
hypergeometric function by using formula (4.434.2) of the hand- 
book just quoted,’ ie., 


Bool) = \, DAR) Gera (8.118) 
—_ p-tnp-ar2 UP + Q/2) on, -at p+ 
gets oe th eo 4 thpth-S), 
where p + q > 0. 


7]. M,. Ryshik and I. S. Gradstein, op. cit., p. 237. 
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4.2. Next, we consider the case where the input process is the 
sum of a Gaussian process ¢(t) with the characteristic functions 
(8.102) and a “‘sine wave”’ 


s(t) = Ecos (wot + go) = Ecos®, 


with fixed amplitude E and completely random initial phase 9, 
(cf. p. 198) which is statistically independent of &(t). Then the 
characteristic function of the total signal 
x(t) = &(t) + s(t) (8.119) 
is just the product of the characteristic functions of the processes 
E(t) and s(t): 
Ceidt — CetMty (einE coe, — en 0707/2 J EQ) : (8.120) 
Here, to calculate <e**), we have averaged over the random initial 
phase g,, using the integral representation of the Bessel function 


Jz). Similarly, for the two-dimensional characteristic function, 
we have 


< ef2uti9,2,% 


= exp (#92 + 192,.€,}) Cexp {iE(2 cos ® + 2, cos P,)}> 


= exp |— ; oA? + 2RQQ, + 2) [LE V2? + 2QQ_ cos wer + 24) 


(8.121) 
[cf. (7.82)]. 

To determine the mean values (7) and <77,>, we substitute the 
characteristic functions (8.120) and (8.121) [instead of the functions 
(8.102)] into (8.106). In calculating <77,), it is convenient to use the 
familiar addition theorem for Bessel functions® 


J{E VPP IO, cos age AB) =F (1) ex J,{EQ) Jy(EQ,)o0s havgr 
kwQ 
(8.122) 


®[. M. Ryshik and I, $. Gradstein, op. cit., formula (6.541.1), p. 335. 
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where 


eae Oe €.=@=..=2. 


We also use the expansion (8.108) for the quantity exp (— o?RQQ,) 
appearing in (8.121). After making all these substitutions, we 
obtain the following double series for the moment <y7,> of the 
output signal: 


3 [Z [ FG) J,(2Q) «tn go)” 

com> = DD [oe J POS) JED) | 
(8.123) 
(=r 


x nl 


€,R” cos Rut « 

To obtain the correlation function &,(7), we have to subtract from 
(8.123) the term corresponding to n = 0, k = 0, which equals 
<n). 


Example 2. Suppose the process (8.119) undergoes the 
transformation 


x for x>0O, 
17-8) 10 for x <0. Cr) 
Then we have 
’ Se 1 
F(iQ) = | ott nds = — oe, (8.125) 


where the contour of integration LE coincides with the real axis 
Im 2 = 0 when the integral has no singularity at the origin 2 = 0, 
and otherwise has a downward indentation avoiding the singularity 
at the origin. According to (8.123) and (8.125), we have 


nt+tk>0 


(8.126) 


x — e, Rr) cos Rwor . 


Only two integrands in this sum have a pole at the origin, i.e., those 
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corresponding to n = 1, k = O and n = 0, k = 1. Since these 
integrands are odd functions of Q, it is easily seen that 


i Jo( EQ) e- 0°"? = = in, 
L 


(8.127) 
1 wore 22 £, 
3 |, EM) ete Ge = sim. 


All the other integrals in (8.126) can be expressed in terms of the 
functions (8.118). As a result, (8.126) reduces to 


2 Ee 
k,() = > R(r) + = 608 eagr 
(8.128) 


am I : 
+ 73 2, sill + (1 HP engi.n-1(E/o) R(7) cos Rargr « 
ny k=aQ ~ 


n+k>1 
‘In particular, (8.128) is valid for a narrow-band random process 
E(t) = A(t) 008 (wot +), 
with the correlation function 
R(t) = r(r) co8 wor , (8.129) 


where A(t), p(t) and r(r) are slowly varying functions, e.g., 
r' 'r < wy, In this case, the output signal 


a(t) = g[A cos (wot + p) + Ecos (wot + )] = g[B cos (wot + ¥)] 
(8.130) 


has spectral components lying in narrow bands near the frequencies 
lw, (J = 0, 1, 2,...), and the correlation function (8.128) can be 
written as a series 


hj(7) = kot) + y(t) CoS wor + Ry(r) cos Zur +4, (8-131) 


involving the slowly varying functions Ao(r), Ay(+), Ro(7), «. 
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Of particular interest is the first function &,(7), obtained by 
averaging k,(7): 
T+(2r / a9) 
| k(t’) dr’. (8.132) 


We now show that in the case of narrow-band signals, the averaged 
correlation function (8.132) is approximately equal to the correlation 
function K[¥%, 7,] of the time-averaged signal 


; wo (eae 


ar eee Oe 
A(t) = 5 n(t’) dt =7 J. n(t-+x)dx, (8.133) 


where Ty = 27/wy,. According to (8.133), we have 
bes og, 1 Ty Te 
K[i, 7 =7 J, ie k(t +x —y) dx dy. (8.134) 


If we substitute (8.131) into (8.134), using the Taylor’s series 


R(r +x —y)= yk (x — y)* C05 MD) css) 


T= 


then, since 


T, pT, 
al *R(7) cos kw(t +x —y)dxdy =0 for k>O 
0 0 


and 
Tg Po 
| | (x — y)’ cos Rwo(r + x — y) dx dy 
0 0 
Te plo ; _ IT 
os a al acer 
we find that 


K[a, a] = 4,(t) + ORT, + RiT) + +). 
It follows that 
k(t) ~ Kl, A, 


as asserted, since the narrow-band condition implies kT, < k,. 


SEC. 4] ‘ZERO-MEMORY NONLINEAR TRANSFORMATIONS 241 


It is most convenient to carry out the averaging (8.132) of the 
expression (8.123) before the expansion (8.108) in powers of R is 
made in (8.121). Thus, expanding only the Bessel function factor in 
(8.121), and using (8.129), we find that 


(ei2zti2,£,) 
= toto lt Y) (—1)¥ ey Jy (EQ) J,(EQ,) €- 792x708%0" cos Revgr. 
k=O 
(8.135) 


We then carry out an average of the type (8.132) with respect to 
7, regarding r(7) as constant; as a result, (8.135) becomes 


e-92(274.02)/8 > (-1)¥ e, J (EQ) J,(E2,) 1,(—o7722,), (8.136) 


where /,(z) is the modified Bessel function of the first kind, of 
order .° Since 


@ l 1 Qn+k 
—g? = i ee 
1,{—0772,) 7 Gen ( 5 or 22,) 


by analogy with (8.123) we have 


23 <o o2ntk . $6 2 
Cite? = By [Aa | FOR) JABO) eee Qae* aa] 
Fase (8.137) 


* “al(n = rCerae ss F ro) 


Subtracting <(7>? = <n>%, the square of the mean value, from 
(8.137), we obtain the correlation function 


k{r) = Roz) 
ae 9192/2 Qentk fore 
la J FOR) {EQ Ae Qs aol ala +B sre] 
ntk>o (8.138) 


since (7? is just the term in (8.137) with k = 0,” = 0. 


"vy, M. Ryshik and I. S. Gradstein, op. cit., formula (6.443), p. 317. 
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If the nonlinear transformation g(x) has the form (8.124), then the 
d-c component (8.133) is proportional to the total amplitude, Le., 


A= 5 f"elBoosy +H dx =, (8.139) 


and hence (8.138) can be used to find the correlation function of the 
amplitude A(t), Substituting (8.125) into (8.138), we find that 


k-(r) = n-*kp(r) 


me ie fm eae: f. Ji EQ) €- 72712 Qantk-2 aa} 
n+k>0 


a@n+k 


fk 
* “ln + Ay 12 57)] 
so that, according to (8.118), 


= >> ad nin aa Lame, engh-y (E/o) ?"**(r). (8.140) 


asks 


CHAPTER 9 


Nonlinear Transformations with 
Memory. Detection of Random Signals 


To illustrate the basic idea of the methods used to study what 
happens when random signals undergo nonlinear transformations 
with memory, we shall consider the example of the diode detector, 
whose action is described by a first-order nonlinear differential 
equation. By the diode detector, we mean the device shown in 
Figure 5, obtained by connecting a diode, i.e.,a nonlinear element 
with current-voltage characteristic J = F(V), to a parallel RC- 
circuit.! Suppose we want to find the statistical characteristics of the 
voltage 7(t) across the RC-circuit, when the input to the detector ts a 
random signal é(t) with known properties. The differential equation 
for the diode detector has the form 


| l 
i+ Re = GFE — 4) (9.1) 


[cf. (8.3)]. A rigorous solution of this nonlinear differential equation 
for typical functions F approximating actual current-voltage 
characteristics is impossible even in the case where the function €(t) 
in the right-hand side of (9.1) is an ordinary (non-constant) 
function of time, rather than a random function. Thus, solving (9.1) 
is certainly all the more unfeasible when &(t) is random, and we 


1 The ordinary diode rectifier, with a purely resistive load, is a simpler 
system, since there is no need to take the reaction of the load into account. 
{n this case, the problem reduces to the study of a zero-memory trans- 
formation of the input process, like those studied in the previous chapter. 
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must have recourse to approximate methods of solution. The choice 
of one method as opposed to another depends on the form of the 
function F(V), on the system parameters, and also on the relation 
between the correlation time 7,,, of the input process and the 
time constant of the detector (equal to RC). We can distinguish 
five special cases, each of which corresponds to a different way of 
handling the problem: 


1. If the inequality | 
Teor > RC (9.2) 


holds, we can study the problem in the quasi-static approximatton. 
In this case, as a first approximation we can neglect the time 
derivative 7 in equation (9.1), and then the problem reduces to the 
zero-memory nonkinear transformation 


9 = RE(E—7)- 


2. Suppose the input process £(t) is narrow-band, i.e., corre- 
sponds to oscillations of frequency w. with slowly varying amplitude 
and phase, so that €() has a large correlation time 7,,,. If the 
relation 


Teor > RC > = (9.3) 


holds, then we can apply the envelope method, which is a variant 
of the quasi-static method. This case includes the detector stages 
used in many radio receivers. 


3. In many cases, it is appropriate to use the method of ‘small 
nonlinearity,” i.e., the problem is regarded as linear in the first 
approximation, and the nonlinear effects are only taken into 
account in subsequent approximations. This method is applicable 
when the nonlinear term (or terms) in the original equation is small, 
or when the equation can be linearized with respect to deviations 
from some suitably chosen zeroth approximation. The application 
of this method js not limited to the case where there is some special 
relation between the time constant of the system and the correlation 
time, and in fact the method can even be applied when 7,,, ~ RC. 
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4. If the time constant is small, which means that 
{ 


Teor RC, (9.4) 


| 

then £(t) can be regarded as a Markov process, and we can make use 
of the Fokker-Planck equation. In particular, this case may arise in 
making measurements, when we want to ascertain the mean value 
<n> of the output signal (characterizing the reading of the measuring 
device) and the variance Dy (characterizing the measurement 
esrqr). Then, it is sufficient to know the one-dimensional stationary 
distribution, which is a solution of the Fokker-Planck equation. 


5. When we are dealing with intermediate values of the correla- 
tion time, i.e., 
Tror ~ RC, 


and when the signals are strong, the problem becomes very 
difficult. If the input signal €(¢) is assumed to be a Markoy process, 
the output voltage will be a component of a multidimensional 
Markov process, and some results can be obtained by using the 
multidimensional Fokker-Planck equation. 

We now study these various methods in turn. 


1. Detection of Narrow-Band Processes. The Envelope Method 


If the input process is slowly varying (Teor S> RC), then, as 
already noted, we can neglect the derivative 7 in equation (9.1), 
obtaining 

] 

9 = RF(£ — 7). (9.5) 
Solving (9.5) for 7 in terms of ~ (for example, graphically), we 
obtain a zero-memory transformation 


n(t) = el&(e)] - (9.6) 


Me:hods for analyzing nonlinear transformations of this kind were 


246 NONLINEAR TRANSFORMATIONS OF SIGNALS AND NOISE [CH. 9 


presented in Chap. 8. It is sometimes convenient to represent the 
function (9.6) as a polynomial, e.g., of the form 


g(é) = a& + a7, 
or 


a(é) == a,é* = a,é* . 


Then, the coefficients a,, a, and a, are chosen numerically in such 
a way that the polynomials give a satisfactory approximation over 
the important part of the transfer characteristic. The statistical 
characteristics of the output signal for these two polynomial 
transfer characteristics were calculated in Examples 1 and 2 of 
Chap. 8, Sec. 2. 


Linear Linear Terminal 


Selective Detector l-f gm- Device 
i-f ampl. plifier 


Fig. 11. Simplified block diagram of a typical radio receiver. 


We now examine in detail a variant of the quasi-static method 
which is suitable in the case where the input signal is a narrow- 
band process. Consider a typical radio receiver (shown schema- 
tically in Figure 11), consisting of a linear selective intermediate- 
frequency (i-f) amplifier, a detector stage, a linear low-frequency 
(l-f) amplifier, and a terminal device. Usually, the i-f am- 
plifier is a frequency-selective four-terminal device, i.e., the 
inequality 4w <w, usually holds, where wy is some “center” 
(angular) frequency, and 4w is the width of the pass band of the 
amplifier. Thus, if the input to the i-f amplifier is a broad-band 
stationary process, the output 7(#) is a narrow-band process, 
which usually has a correlation function of the form 


R(t) = 1(r) 008 wor , 


where 7(r) is slowly varying compared to cos wor. [In formula (7.4), 
the function s(r) vanishes because of the symmetry of the frequency 
response of the i-f amplifier about the center frequency wo.] 
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Like every narrow-band process, é(f) can be written as an 
oscillation 
| 


E(t) = A(t) cos [wot + o(t)], (9.7) 


with slowly varying amplitude and phase (7.55). If in addition to the 
process £(t), the input to the detector contains a sine wave E cos wot, 
then the total signal &¢) + E cos wot can also be written as an 
oscillation of the form (7.83), with a different amplitude B(t) and 
phase y(t). Thus, the results to be obtained below can be extended 
to this case, if we simply replace the amplitude A(t) by B(t). 

’ The function of any detector is to “‘extract” the low-frequency 
modulation from its input in as effective a manner as possible, and 
hence the output voltage across the RC-circuit must manage to 
follow” the amplitude changes. Since the amplitude varies with 
the time constant 7,,,, this means that the condition 


Teor > RC (9.8) 


must hold. On the other hand, the detector must exert a strong 
smoothing action on the radio-frequency (r-f) oscillations, which 
means that the condition 


RC> an (9.9) 
Wa 


must hold. If these two conditions are satisfied, the study of the 

detection problem is greatly simplified, since then, as we shall see, 

the output voltage 7(t) is related to the amplitude A(t) [or B(t)] of 

the input oscillation by a zero-memory transformation. 
Substituting (9.7) into (9.1), we obtain 


l 1 


Integrating (9.10) over a period from ¢ to t + (27/wg), we find that 


| Or 
nlt + —) — n(t) 
( = (9.11) 
24 f +(27/w,) {—7(t’) + RF[A(t’) cos (wot’ + ~) — 7{t’)]} dt’ . 


i ROL: 
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However, because of (9.9), the function 7(t) only manages to 
change very slightly during one period. Therefore, the difference 


a(t + =) ~ x(t) 


is almost equal to (27/wo)7, and 7(t’) can be regarded as constant 
in the integrand, i.e., n(t’) ~ 7(t). The amplitude A and the phase 
can also be regarded as constant during a period, so that we need 
only take into account the change of cos [wot’ + ¢(t)]. Thus, (9.11) 
reduces to 


; 1 1 ty (22 / wo) ; 
1+ ROT = Goel, FA cos (wet’ + @) — 7] dt 
or 
ee ea fe i eee 9.12) 
q RO? = Cis], [ Os x ny aX - (9. 


The right-hand side is essentially a known function of A and n, 
obtained by integrating the current-voltage characteristic F(V) of 
the diode. 

As it stands, equation (9.12) is hard to solve in general form. 
However, the problem is greatly simplified by the condition (9.8) 
which requires that the amplitude vary slowly; in this case, we can 
neglect the derivative 4 in (9.12). In fact, just as in the case of A(t) 
itself, the only important spectral components of the process 7(t) 
lie in the frequency band 

| 


Teor 


|w—wy|~ dw = 


For these frequencies, the derivative 7,, = iw, is much smaller 
than 7,,/RC, because of (9.8). Therefore, the derivative 7 in (9.12) 
is much smaller than the second term 7/RC. If we neglect 7, 


(9.12) becomes 
on 
I=, | F(A cos x — 7) dx, (9.13) 


which specifies the zero-memory transformation leading from the 
amplitude A(t) to the output voltage 7(t). 
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We now use (9.13) to study three typical cases of diode detection, 
where the current-voltage characteristic F(V) is piecewise linear, 
quadratic and exponential, respectively. 


Example 1. In this case, usually called the linear detector, the 
current-voltage characteristic is 


we for V>Q, 


1=F(V) = §R, (9.14) 
0 for V<O, 


where R, is the internal resistance of the diode in its conducting 
state. Substituting (9.14) into (9.13), we have 


_ AR fv 7 
where 
y = arccos 4. 


We now introduce the dimensionless quantity 


k=, (9.16) 


which can be called the ‘‘envelope reproduction coefficient.” 
Then (9.15) becomes 


k= 


rs 


R arc cosk 
R, I. (cos x — R) dy, 


and carrying out the integration, we finally obtain 


R wR 


R, Vi—# —karccosk’ 


(9.17) 


If R/R; <1, the envelope reproduction coefficient & is < 0.3, 


and then (9.17) implies that 
1R 


sede ae 
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However, such values of R/R; are not encountered in typical radio 
receivers. In fact, this ratio ts of the order of 3 to 10 in radar and 
television receivers, and can equal several hundred in communi- 
cation receivers. In Figure 12, we plot the coefficient & for large 
values of R/R;. It is clear from the figure that the envelope repro- 
duction coefficient k grows monotonically with R/R,;, and 
approaches | asymptotically. 


l.2 


pth] ot 
Pe Ms SS 

eee hh 
0.4 Bes = 

a 0 ee 

0 ie ae A a 

| {e) exe) lo00 «CR. 

R. 


Fig. 12. Dependence of the envelope reproduction coefficient k on the 
resistance ratio R/R, for the linear detector. 


The characteristic feature of the linear detector, as opposed to 
other kinds of detectors, is that its output voltage 7(t) depends 
linearly on A(t), and therefore it is appropriate to introduce the 
envelope reproduction coefficient (9.16), which does not depend on 
A. After calculating 2, the statistical characteristics of the output 
signal 


A(t) = RA(t) (9.18) 


can be found very simply from the corresponding characteristics of 
A(t). In particular, we have 


<= RAD, (mp = RXAAD, 


where it is useful to recall that <4A,) is given by the third of the 
formulas (8.56). 
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’ Example 2. For the quadratic detector, the current-voltage 
characteristic is given by 


BV? for V>O, 


EY NOs ioe VO: 


(9.19) 


where the coefficient 8 is a dimensionless constant. Using (9.13), 


we obtain 
Et binant 


where 
7} 
= arc cos —, 
Y A 


as before; again introducing k = 7/A, we have 


\ arc cos k 
se (cos x — AR dy. 


ae a 


Carrying out the integration, we obtain the following  tran- 
scendental equation, after making some elementary calculations: 
: 2nk 


LAS 9.20 
: (1 + 2k?) arccosk ~ 3k Vi —#? (9.20) 


In this case, the envelope reproduction coefficient & no longer has 
its former meaning, but depends on A. 


If BRA < 0.1, the inequality <1 holds, and then, setting 
arc cos k = 7/2, we find from (9.20) that 


kw4BRA, 748RA?. (9.21) 


For large values of BRA, the coefficient & can be found by solving 
equation (9.20) numerically. In Figure 13, we plot the function 


k = f (BRA), (9.22) 


where f(x) is the inverse of the function 
4 


Ink 
a (1 + 2k) arccosk — 3k-V/1 ~~ 
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As the figure shows, the coefficient k grows monotonically with 
BRA, and oe 1 asymptotically. 


Fig. 13. Dependence of the coefficient k on the parameter BRA 
for the quadratic detector. 


It follows from (9.22) that the dependence of the output voltage 
on the amplitude can be written in the form 


BRn = BRA f (BRA) . (9.23) 


The function (9.23) can be approximated by a polynomial in some 
region or other, depending on the range of values of BRA most 
frequently encountered, For example, the quadratic polynomial 


BR» = 0.583(BRA) + 0.002(8RA)? 


plotted in Figure 14 gives a satisfactory approximation to (9.23) 


BRn 
80 


60 
40 
20 
e 50 100 BRA 


Fig, 14. Approximate dependence of the output voltage on the 
amplitude for the quadratic detector. 
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in the range BRA = Oto 100, but it is best to use this approxima- 
tion when BRA ~ 50, since it is not too accurate for BRA < 50. 
For amplitudes corresponding to a different range of BRA, one 
should use a polynomial with different coefficients. After choosing 
the appropriate polynomial, the statistical characteristics of the 
output signal can be calculated by using the method given in 
Example 2 of Chap. 8, Sec. 2. 


: Example 3. Finally, we consider the exponential detector. In 
this case, the current-voltage characteristic has the exponential 
form 


J 


I = F(V) = ier”, (9.24) 


where i, and a are constants, J is the current through the nonlinear 
element, and V is the voltage across it. The curve (9.24) can be used 
to approximate the current-voltage characteristics of actual diodes. 
with 
ty == 50 to 600 microamperes, 
a = 5 to 10 volts", 


in the case of oxide-coated cathodes at normal working tem- 
peratures. 

If the input to the detector is the process (9.7), then, according 
to (9.13), the voltage across the RC-circuit is given by 


! 


soe pee ; 
7 = Rt an i exp {a(A cos xy — 9)} dy = Riye-9" I(aA) 


or 
ane*? = ARil,(aA) . (9.25) 
Here I,)(z) = Jo(éz) is the modified Bessel function of the first 
kind, of order zero, Equation (9.25) can be solved in various ways, 
dexending on whether the quantity aRJ,(a(A>) or the quantity 
a¢ A> is large or small compared to unity. 
If a(A> <1, then 7 differs very slightly from the constant 
voitage 7, which appears in the absence of any input voltage, and 
corresponds to setting A = 0 in (9.25): 


anget = aRin. (9.26) 
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Using (9.26) and the expansions 


— pal) yagy 1 ean 
[(aA) = 1 + 4 (aA)? + 6d (aA)t + ..., 
xe? = xyeto 4 (1 + xq) e0(x — my) + (2 — x), 
we find from (9.25) that 
aq 
— | ~~ a ee aA* ‘< 52 
7 No 4(1 As a7) (9 7) 


Therefore, for small values aA < 0.1, the voltage across the 
detector load due to the presence of the input signal is proportional 
to the square of the envelope. In fact, we have quadratic detection 
quite generally for small input amplitudes, regardless of the form 
of the current-voltage characteristic F(V). This is related to the 
fact that the output signal is an even function of A, since it does 
not depend on the phase of the input signal, and does not change 
when the phase is shifted by 180°, i.e., when A is replaced by — A. 
Thus, only even powers of A appear in the power series expansion 
of 7 — 7, aud hence for small values of A, only the first term of 
the series, proportional to A?, is important. 
The case where ARz, and aA are both large, 1.e., 


aRigl (aA) > 1, (9.28) 


is of great practical interest. Then the envelope reproduction 
coefficient 7/A is large, of order unity. Using the condition (9.28), 
we can solve equation (9.25) by the method of successive approxi- 
mations. Writing (9.25) in the form 


x+tiIinxs=2, 
where 


x= an, x = In [aRiI,(aA)], 
we define the successive approximations by 


4, = =z — nx, (t= 1525.0) s 
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Oy 
xy = 2, 
OG %g = z—Inz, 


wists inajae ine ee, 
x 


Confining ourselves to the second approximation, we have 
d 


: l aRiy{,(aA) 


7 a Tn faRigl (aA) (9.29) 


Next, we use (9.29) to determine the coefficients a, and a, in the 
expression (8.42), choosing c to be <A), the mean value of the 
amplitude: 

7 = a +a,(A — <A>) + a,{A — CAD). 


Equating the terms in this polynomial to the corresponding terms 
of the Taylor’s series expansion of (9.29), we obtain 


a, = —(% — Inz), 
a, =(1—-+)e, (9.30) 
a # a ! 3 
m= 5-2 t5(l-s)('-aay - 9), 
where 
2, = InaRi, +InI(acAd), 9 = pee 


As we increase the parameter a¢ A), 
BO, =>], 


so that, according to (9.30), the coefficient a, > 1, while a, — 0. 
Tus, in this case, the transformation of the signal by the exponen- 
tial d stector approaches the ideal detector 

n=a+A (a =a,—<Ad), (9.31) 


which reproduces the envelope exactly. 
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For an arbitrary current-voltage characteristic F(V), the more 
steeply the function F(V) grows compared to V/R, 1.e., the more 
valid the condition 


R,<R, where R, = and V>O, (9.32) 


I 
OF IOV 


the closer the detection is to being ideal. 


2. The Method of Smal] Nonlinearity 


We shall present the other methods of analyzing nonlinear 
transformations with memory by applying them to the special case 
where the current-voltage characteristic of the nonlinear element 
(diode) is an exponential function 


F(V) = iget” ; (9.33) 


here, i, and a are the constants discussed earlier (p. 253). Then 
equation (9.1) becomes 


4+ pe" - 2 ers (9.34) 
It is convenient to introduce the new variable 
x= em, (9.35) 
which satisfies the equation 
RC% + xInx = atyRe**. (9.36) 


The character of the process x(t) satisfying (9.36) is determined 
by the size of the parameter aigR, and also by the ratio of the 
correlation time 7,,, of the process €(t) to the time constant RC. 
Depending on whether these parameters aipR and r,,,/RC are 
large or small, we can apply various special asymptotic methods for 
studying x(t). For example, the condition 7,,, S> RC allows us to 
use the quasi-static method. On the other hand, if 7,,, << RC, we 
can use stochastic methods involving the Fokker-Planck equation. 
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Iiowever, neither of these methods is suitable if 7,,, ~ RC. This 
compels us to have recourse to another method, which is appro- 
priate when the other parameter az)R satishes certain conditions. 
For the time being, we postpone the precise formulation of these 
conditions, and instead explain the method itself, which we call the 
method of small nonlinearity. In this method, we look for successive 
approximations to the solution x(t) of equation (9.36), The zeroth 
approximation x,(t) is defined as the solution of the equation 


which is suggested by averaging (9.36). Assuming that &t) is a 
stationary process, we are interested in a stationary process x(¢) 
which began in the remote past. This means that a stationary 
‘value of x) has been established, which, according to (9.37), 
satishes the equation 


x, In xp = ai,R<er) . (9.38) 
‘The average <e**) is the value of the characteristic function 


<e™» for u = —ia. If &(t) is a Gaussian process with zero mean, 
then 


Keluby = enon? /2 <etty = 2707/2 | (9.39) 
and (9.38) becomes 
Xo IN xo = aiyRero/? , (9.40) 
Using (9.35), we find that 


l 

To = 3 In x9 
satishes the equation 
; e770 = i,Ree"e*/ rast (9.41) 


‘ ‘Next, we derive a differential equation for the deviation 
; 


i 6 ae eee (9.42) 


Subtracting (9.37) from (9.36), we find that 


| RC 4- (%q + 2) In (x9 + 2) — x9 IN x9 = aigR(er — <e*®). (9.43) 
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If the function x In « can be expanded in a Taylor’s series 


l 


BADE aie Mer Ute tel) Bee gaa 3 (9.44) 
then equation (9.43) becomes 
: 2 = 
RC + (In x) + 1) 2 = ai RE — = Ae oa (9.45) 
where 
{ = et§ — cert), (9.46) 


This equation contains the nonlinear terms 


- — a3 4-44. = f (Xp + 2) — f (Xp) —f'(%) Zz, (9.47) 


where 

f(x) = xInx. 
The use of the method of small nonlinearity is based on the 
assumption that these nonlinear terms have comparatively little 
effect, and can be neglected in a first approximation. Therefore, 
(9.45) leads to the following equation for the first approximation: 


RCz, + (In xp + 1) 3, = @typRE. (9.48) 


To clarify how higher-order approximations are constructed, we 
introduce an auxiliary small parameter «. Then, we write the 
solution (9.42) as an expansion 


giesese Pets to ete: tt, (9.49) 


and we equip the nonlinear terms with appropriate small para- 
meters. ‘Thus, writing (9.45) in the form 


RCz + (Ina + I) 2 = aight — £2? + Sx, 
0 0 


substituting from (9.49), and equating terms of the same order in «, 
we find that the equation for the next approximation is 
zi 
RC#_ + (In % + 1) z= ~ 2h (9.50) 


2X9 
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When it is inconvenient to make the ‘T'aylor’s scries expansion 
(9.47), or when the expansion does not exist, the whole nonlinear 
term (9.47) appearing in (9.45) can be multiplied by a small 
parameter. With this version of our method, the whole nonlinear 
function 


F (Xo + 1) —f (%0) —F'(X0) 21 


will appear in the right-hand side of the equation for z,. Once the 
method of constructing the various approximations is clear, we 
can get rid of the small parameter, by setting it equal to I. 

The right-hand sides of (9.48) and (9.50) involve only known 
functions or functions already found in lower-order approxima- 
tions. Moreover, these equations are linear in the unknown 
functions, and can be solved without any particuler difficulty. 
Thus, the stationary solution of equation (9.48) is 


2,(t) = i exp - inte tN —t’)) Ut’) dt’. (9.51) 


According to the definition (9.46), the process ((t) has mean value 
zero, and hence 


(z,) = 0. (9.52) 


We now find the correlation function and the variance of the 
first approximation z,(t). First, we calculate the correlation function 
of the process {(t), assuming that €(¢) is a stationary Gaus- 
sian process with mean value zero and correlation function 
k(r) = o?R(r). Because of (9.46), we have 


KOC» = Cetttatry — Kem? (9.53) 
The quantity <e***"*:) can be expressed in terms of the two- 
dimensional characteristic function, given by the familiar formula 


(8.102). Thus, evaluating @,(—za, —ia) and using (9.39), we obtain 


<tt> —_ e2%a"[ea’otRit) —_ 1] : (9.54) 
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According to (9.51), the correlation function 2,2z,, equals 


Caen? = (SY f . { - er Mth) -BIt+E-t) CL (t4) L(t9)> dty dle, (9.55) 
where 
Making the change of variables 
p=t,—t,—T, pat, 


we can write (9.55) as 


Aly 


(Br) = ( Cc Y {_ dp eae e~ Piat—ae4rh (EC > ds. 


After evaluating the integral with respect to s, this becomes 


oe 
Canad = (FEY 3g fe tbo dp (9.56) 


In particular, setting 7 = 0, we find the variance 


; 2 1 foo} 
> = (A) pf emus» ap. 0.57) 
Using (9.54), we obtain 


2 _ (%RP 2gr | Zo*R(z) __ — Bt 
GD =p eT" Re, (e2 lye dr. (9.58) 


To obtain more exact results, we can substitute (9.51) into the 
right-hand side of (9.50), and then calculate a higher-order 
approximation. 

The statistical characteristics of the output voltage 


l 
7 = 7 ln (% + 2) 
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can now be found by expanding 7 in a Taylor’s scrics 


ry 


2 & 
= No + ax, ee 2ax? + ou, (9.59) 


and then substituting the series (9.49) into (9.59). If we confine 
ourselves to results involving the first approximations (9.51), 


(9.56) and (9.57), we have 


<7 = No — wt dat) : 


2ax? 
1 (9.60) 
«(7 — 79) (7, — %)> = a C32; 
0 
and in particular, 

l 

Da = ae (9.61) 
1] 


To determine the right-hand sides of these expressions, we use 
formulas (9.56) to (9.58). 

We now analyze the conditions under which the above mcthod is 
applicable, for the most difficult case, i.c., where the process £(t) 
has a correlation time of the order of the filter time constant 
(Teor ~ RC). The method of small nonlinearity is applicable if the 
nonlinear terms in equation (9.45) have a small effect compared to 
the linear term (In x) + 1)z. Choosing the first of the nonlinear 
terms, we obtain the condition 


ge 
| (Ina + 1)z|> ra 


or 


| z| <2x {In x9 -4- 1) (9.62) 


where for z we can take the first approximation (9.51). According 
to (9.38) and (9.40), the quantity x,(In x, + 1) differs only slightly 
from 

Xp In x9 == Qin Re®*?? ~ ain Re-*2"! off) , (9.63) 


where o(€) is the standard deviation of f, since according to (9.54), 


o(f) ay eta? : 
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On the other hand, using (9.51) we can obtain the estimate 


ai, _RC 

as C Inn +1°’ (9.64) 

where the inequality sign corresponds to the case where the 
function {() changes rapidly, with time constant 


RC 


Teor S ins, 1° 
In the opposite case, where 7,,, ~ RC, the quantity 2, is chiefly 
determined by the factor e~*'"? appearing in the integrand of 
(9.51), and then the sign ~ should be chosen in (9.64). 
Substituting (9.64) into (9.62), and taking into account the 
estimate (9.63) and the relation { ~ o(£), we find that 


In x9 > e2°*/? , 


Multiplying both sides of this inequality by x9 and using (9.40), 
we have 
aigR > xp (9.65) 
or 
@iyR In (atgR) > xo In xq. 


Using (9.40) again, and dividing by at)R, we write the small- 
nonlinearity condition in the form 


In (ai,R) > e2°2*/? , (9.66) 


Thus, the method of small nonlinearity is applicable for large 
values of the parameter ai,R and for noise levels which are not too 
large. In the case of a small correlation time 7,,,, corresponding to 
the sign < in (9.64), we can replace (9.65) and (9.66) by less 
stringent conditions. Then, the region in which the method of 
small nonlinearity is applicable, which is essentially determined by 
the condition o(x) < x», becomes somewhat larger. 
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3. The Method of the Fokker-Planck Equation 


When the input process has a sufficiently small correlation 
time, i.e., one which is smaller than the other time constants of the 
system, we can analyze the process x(t) by using stochastic methods 
involving the Tokker-Planck equation. We shall illustrate this 
method by applying it to the previously considered problem of the 
exponential detector, described by equation (9.36), which we write 
in the form 


RCx 4 xInx = ayyR¢et*> -) atyRC. (9.67) 
When 7,,, is small, we can replace the random perturbation 
C = et -- <ert> (9.68) 


in (9.67) by a Gaussian delta-correlated random process ¢, which 
has mean value zcro (like 2) and correlation fuuction 


CEL) = K&(r). (9.69) 


Here, the intensity coefficient K is chosen equal to the intensity 
coefficient of the original process (9.68): 


K= ig CLL, dr. (9.70) 


After we have made this replacement of { by €, equation (9.67) 
describes a Markov process, and is cquivalent to the Fokker-Planck 
equation 


_ | ra) Aly, Cen 
w = RC py [ee Ine — me] + 5 L (a nee (9.71) 
where 
m == ai,Riet® > = aiyRe? "i? (9.72) 


[ef. (9.39)]. 
Finding the nonstationary solution of cquation (9.71) ts a 
complicated problem, but the one-dimensional — stationary 
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probability density w(x) can be found quite easily. Applying (4.49), 
we have 


w(x) == N-! exp iS GK [:2(In x— ;) ~ 2ms|| » (9.73) 


where N is a suitable normalization constant. It is clear that the 
form of the distribution (9.73) is determined by only two parameters 


m = aipRe*o"/? and g = AGokY K. (9.74) 
The first parameter m determines the coordinate x) with maximum 
probability, and in fact x is just the quantity satisfying equation 
(9.40), i.e., x9 ln x9 = m. The second parameter q determines the 
amount of ‘‘scatter’”’ due to the fluctuations. Thus, the coordinates 
x, and x,, for which the probability density falls off to e~!/? ~ 0.61 
times its maximum value, can be found as the roots of the equation 


F(x) —f(%) = 9, (9.75) 
where 
f(x) = xX{In x — $) — mx. (9.76) 


From the distribution (9.73), we can easily find the probability 
density of the output voltage 7, which is related to x by the zero- 
memory nonlinear transformation 


l 


y= A Inx. 
Using (9.73) and formula (1.15), we obtain 
] 
wn) = ~ exp \- 27 [e2n(an ~2)- 2meen| + an} » (9.77) 


Because of the presence of the term a7 in the exponential, whose 
appearance is due to the derivative dx/dyj, the value 7,, corre- 
sponding to the maximum of the probability density (9.77) does not 
equal 


=-Inx,. 
No a 0 
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In fact, 7, is now defined as the rvot of the equation 
ane*21 — met? —@ --- 0, (9,78) 


Making a Taylor’s series expansion of the furtction which is the 
argument of the exponential in (9.77) and takes its maximum value 
at 7 = 7,,, we obtain 

I 


~ 5p feta — 4) ~ Ime") 4. 


~~ 5 [eam — 8) ~ Ime™ra] 1 ay (9.79) 
cs sp [etm 2m + 1) — mem] aX — 2 | 
Retaining only these first few terms, we find that 
rn) © an) exp | — 5 | (9.80) 


in the region of maximum probability. Here, 


Of0 as 4 [6?0%m(2amm + 1) — mee mt 
(9.81) 


| bart 
= $0-+ gh) ements sh) 
[cf. (9.78)}] is the variance of an equivalent Gaussian distrihution 
which is close to w(7) in the region of maximum probability. 

If the distribution w(y) differs only slightly from a normal 
distribution, the quantity 7,, is approximately equal to the nican 
value <7), and the equivalent variance o°, is approximately equal 
to the variance Dy. For distributions which are not approximately 
normal, 7,, and o2, are no longer close to <7) and Dn, respectively, 
but they still serve to characterize the distribution z(7). ‘The 
equivalent variance o2, characterizes the amount of “fluctuational 
scatter,” in the sense that Dy ~ o3,. If we want to determine the 


actual mean value and variance in the case where w(7) is not 
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approximately normal, we can calculate the appropriate integrals 
involving the weight (9.77), resorting to numerical integration if 
necessary. 

For Gaussian noise &(t), with 


<> =0, Eb = oF R(z), 
we have 
K = Destot [ festo'Ro) 1) dr (9.82) 
0 
according to (9.54) and (9.70). For some specific forms of the 


correlation coefficient R(r), we can immediately calculate this last 
integral. For example, if 


a (9.83) 


Rr) = ev"), y= 
then, making the change of variable 


z= aorev' , 


we find that 


K= < est i. = = a ee Seats [E*(a%o?) —2lnao — C]. (9.84) 
0 
Here, E*(z) is one of the exponential integrals,? and C = 0.577... 
is Euler’s constant. If the integral (9.82) cannot be calculated 
directly, the expansion 
K = 20 5 (ao { ” RM 2) dr (9.85) 
: 0 


| 
nal n 


may turn out to be useful. 
As an example, suppose that the parameters have the values 


= 100. 


ao = 3, aiyR = 10, yRC = 


Teor 


2 See Bateman Manuscript Project, op. ctt., p. 143. 
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Then 
m = 898 , g -- 8.25 > 10%, 


and, using (9:41), (9.78) and (9.81), we find that 
ano = 5.16, an, == 7.05, aay = 0.655. 


On the other hand, if ao and azyR have the same values, while the 
capacity is increased }0 times, so that yRC -- 1000, then 


q = 8.25 x 105, an,, = 6.1, 40 cq += 0.543. 


By comparing these results, we see that when the capacity C is 
increased, both the mean value of the voltage across the load and its 
fluctuational scatter decrease. 

From the above quantitative results, pertaining to Gaussian noise 
with the specific correlation function (9.83) we can also conclude 
that the readings of a voltmeter which measures the mean value of 
the output voltage will be different for different valucs of 7.,,/RC, 
even if the variance o? of the input noise remains the same. There- 
fore, it must be admitted that the use made of vacuum-tube 
voltmeters in ordinary engineering practice, to measure noise 
variances, is erroneous. Vacuum-tube voltmeters can be used only 
for comparative estimates of noise intensities, and only for noise 
whose spectral composition does, not change. ‘Thermoelectric 
devices should be used to measure actua] mean square valucs of the 
noise fluctuations. 


4. Detection of a Sine Wave Plus Noise with a 
Small Correlation Time 


The method for calculating the one-dimensional distribution of 
the output of a nonlinear device with memory, presented in the 
preceding sections, is also applicable when the input contains 
a “sine wave” Ecos wot, together with the random noise &t). 
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Then, for a detector with the exponential characteristic (9.33), the 
corresponding version of equation (9.36) is 


RC% 4+ xInx = aR exp {aE cos wt + a€}. (9.86) 


We now consider two special cases: 


1. Let the input noise process &(t) be slowly varying compared to 
the rapidly oscillating signal E cos wot. We assume that the period 
Ty) = 2n/w, of these oscillations is much less than both the 
correlation time 7,,, of the noise and the filter time constant RC. 
Then equation (9.86) can be averaged over a period, which gives 


RC& + xInx = atyRI (aE) e** . (9.87) 


This equation differs from (9.36) only by having the parameter 
ai,RI,(@E) instead of ai)R. Therefore, with the corresponding 
changes, everything said in Secs. 2 and 3 applies to this case. 


2. The case where the process £(¢) is narrow-band is somewhat 
more distinctive. We assume that the frequency band of &(t) lies 
near the frequency wy of the sine wave. Then, the total input signal 
can be represented as an oscillation with resultant amplitude A(t) 
and phase y(t), Le., 


E cos wot + &(t) = B(t) cos [wot + Y(t). (9.88) 


We have already encountered such signals in Chap. 7 [cf. formula 
(7.80)). If &(¢) is a Gaussian process, then the probability density of 
the amplitude B is of the form (7.88). 

Substituting (9.88) into (9.86), we can use the fact that B(t) and 
p(t) are slowly varying compared to a sine wave of frequency wo. 
This allows us to average the right-hand side of (9.86) over a 
period 7, = 2m/w 9, by regarding B(t) and s(t) as constants. The 
result is 

RCx + xInx = a1,RI,[aB(t)] . (9.89) 
When the time constant RC greatly exceeds the time in which the 
amplitude B changes, i.e., 
Teor GRC, 
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where 7,,, is the correlation time of the noise, we can apply 
stochastic methods involving the Fokker-Planck equation. ‘Thus, 
we regard the ‘‘fluctuational component” 


O(t) = Io(@B) — <1,(283)> (9.90) 
in (9.89) as being delta-correlated, by making the change 


(O61) -> Ka(z7), (9.91) 


where 


K=[ Qbydr (9.92) 


is the intensity coefficient of the process (9.90). ft is convenient to 
write 


m == atgRCI{aB)> , (9.93) 
since then (9.89) has the same form 
RCx + xInx =m +ai,RE (9.94) 


as in the preceding section, so that formulas (9.73) and (9.77) to 
(9.81) can be applied to the present case without any changes at all. 

The parameters (9.92) and (9.93) of the process ¢(f) ivolve the 
averages (/,(aB)> and </,(aB)J,(aB,)>. To calcntate these quantities, 
it is convenient to average the random function 


eaztt) = exp {E cos wot + a€(1)} 


before carrying out the time average over a period. hus, if €(t) is a 
Gnussian random process with mean value Zero and corrclation 
function 
o?R(r) = 0°11) Cos ay7 , 
we have 
e%) = exp {aE coswyt |- dae}, 
< 7 Pp { 0 2 t (9.95) 


Ceatt az) 


= exp {aE cos wot + aE coswft -+ 7) * aol | r(7) cos oz}. 


270 NONLINEAR TRANSFORMATIONS OF SIGNALS AND NOISE [CH. 9 


Averaging the first of these expressions with respect to ¢ over a 
period, we obtain 


34 (250/09) 


<1,(@B)> = = | Cearit’) dt! == ea*0*/2 1,(aE). 
t 


Averaging the second of the equations (9.95) with respect to t over 
a period ts equivalent to averaging the expression (8.121) with 
respect to a random initial phase. Setting 12 = 182, = a in (8.12}) 
and (8.135), we find that 


Cerrar = evo? > e P(ak) et?s2rC08w9t Cog ket ; (9.96) 
A-0 
We now average (9.96) with respect to 7. According to (8.136), this 
gives 
er’a? 5 e,1i(aE) I,{a°o"r) . 
py wf(ak) I,(ao'r) 


Therefore, we have 


CIaB) I(aB,)> ~< et? > «,1( ak) 1,{a°o?r(z)) , (9.97) 
A=0 


KCC, > = e? [ ak) {Iyp[a°a?r(r)] -- 1} 4 Zea? SY T(aE) I,[a’o*r(7)] . 
(9.98) 


To calculate the intensity coefficient K, we substitute (9.98) into 
(9.92). It may be convenient to use a series expansion of the Bessel 
function, when evaluating the integral with respect to 7. Suppose 
the correlation coefficient is of the form 


r(r) = evlylr, 
Then we find it useful to introduce the new variable of integration 
z —~ adore)". 


In the special case EF == 0, according to (9.98) and (9.92), we have 


~ 2 age (7 A(z) at SS a’o” \2n 
a oe ar Zane | 7) + (999) 
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When ao? 1, it ts convenient to use the asymptotic represen- 
tation 


[a - AG) 4-2-3 Ae =) | -}- n} 2% fa, el 1... (9.100) 


It is interesting to compare the expression (9.99) with formula 
(9.84), pertaining to the case of slowly varying input fluctu.tions. 
Tor the value ao = 2, the intensity coefficient (9.84) is 


while, for ‘‘quasi-harmonic”’ input fluctuations, (9.99) gives 


— 


K = eg 42 +059 +011 10.03 + ...)s 7°. 


a 


5. The Case of intermediate Correlation Times 


The most difficult case is where the correlation time of the input 
process &(t) in the equation 


; l l 
71 par= GF - (9.101) 


is comparable with the filter time constant RC. ‘Then, we can use 
neither the quasi-static model, nor the method of the lokker- 
Planck equation in the form given aboye. However, in this case, we 
can stil} use stochastic methods involving the Fokker-Tlanck 
equation, if we consider two-dimensional Markov processes. 

Thus, let (t) be a Gaussian process with mean valire zcro and 
correlation function 


(Ebr, = oe rll, (9.102) 


As we know from Chap. 4, Sec. 10, such a process is Markovian 
and satisfies the differential equation 


E+ yé— Ut), (9.103) 
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where {(t) is a Gaussian delta-correlated process such that 
Cs -50, bby = 2y0%8(r) = K,S(z) - 


Equations (9.101) and (9.103) together specify a two-dimensional 
Markov process. Writing v= €— 7, we can replace these 
equations by the new system 


v I l 
v= — Re- GIO) —(y— RAE tS 
(9.104) 
are tc yvé =e 4 , 
with the equivalent Fokker-Planck equation 
, _ Olfpu I I ow 
= SollRe + CPO] + (y — Re) Fae 
(9.105) 


a o*w Ctw fw 
+ yg (64) -+ yo? (ss + 2 B08 mi rd, . 


Using this equation to find the probability density wv, &) or 
wv, €; v,, €,), We can then integrate with respect to & and &, to 
obtain 


wn) = | é — 7, €) 4& , 
w(n, 7.) = { w(E — 9, £5 & — ar, €,) dE dé, . 


Unfortunately, the complete solution of equation (9.103) is a 
difficult problem, and hence we shall confine ourselves to special 
cases. 


5.1. First, we consider the case where the correlation time is 
exactly equal to the filter time constant, so that 


l 
Then the voltage across the diode satisfies the equation 
pa dey ty 9.107) 
Ce Re Ge) 0. 
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and since it docs not contain £, this equation can be solved indepen- 
dently. From the process z(t), we can obtain the output voltage by 
using (9.10}). The result is 


Bee ear ene eee Ce a ee 
Te got ag Rs, IS) 
since 
l _ 
at Re? ae 


The average of the stationary value of » equals 
<n = R<F(e), , (9 109) 
and to find the output correlation function, it is suffictent to know 
CFD) F(,) 
The Fokker-Planck equation corresponding to (9.107) Is 


2(v) = ae {fv + RF(e)] et | yo? s i) (9.110) 


which has the stationary solution 
v2 Rr. 
wv) = ar exp | — ar carf Feydt. nh 


To find (F(v)), we have to evaluate the integrals 


l Pre 4 ; \ ve R v mf oe } fo. 
FO) = = “xP | Jot — Gt I. Wee Eleyale” 2 CN12) 
and 


N = f- exp _ or — 4 io F(z) ds! do. 


If the current-voltage characteristic of the diodé 1s piecewise linear, 
of the form 


Se for v>-O, 


Fe) = |p for v <0, 9.113) 
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these formulas Icad to the results 


(9.114) 


N= Jr0 KL F RS)-!P), 


and hence, according to (9.109), 


2 RSo 
eee ee 9.115 
V2 14 RS4V14RS ie12) 


We get different expressions for <(F(v)), N and <n) if the 
characteristic /'(v) has a different form. In what follows, we shall not 
specify the precise form of the function /(v), but we shall assume 
that it vanishes for v <0, and grows rapidly for v > 0, in fact, 
much more rapidly than v/R. This second condition is satisfied in 
most practical cases, where the diode has a small internal resistance 
R; < R in its conducting state, so that the ratio of the output 
voltage of the detector to the input voltage is close to unity. Under 
these conditions, the probability 


_ 
Wy == | w(v) dv — ss a (9.116) 


that the diode is in the nonconducting state greatly exceeds the 
probability 


x | w R v 
WwW. — {. w(v) dv =~ W le exp ee [ Fe) dz| dv (9.117) 


that it is in the conducting state. Moreover, the inequality 


WwW. <2, (9.118) 


shows that most of the time the diode is in the nonconducting state. 
In fact, it is clear that 


w, wl, Ween (9.119) 
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To calculate the average (9.112), we can neglect the term 7/2 fn the 
exponential compared to R fj. f(s) dz. obtaining 
= 2) x Ri oe Vice. es j2o 
<F(v)> = V2tf exp i7 3 is F(z) dz, F(x) dv .- ViR 
(9.120) 


<q> = nee 
WT 


Turning to the calculation of the correlation function, we fix 
the initial position v(0) = vw of the “representative point” (see 
p. 63). Suppose that v, < 0. Then, according to (9.107), we have 


Otyv = Ut), (9.121) 


as long as v is negative, since the function F(z) has no effect in the 
region where wv is negative. Using (9.121), we can find the average 
displacement and the “‘fluctuational scatter” of the representative 
point: 


Cup = ve", 
(9.122) 
Du = o%] — e-*7). 
Therefore, v(t) is described by the-probability density 
eee eh MY ee 
MOUUE TS re ee tee ie 
which is the solution of the Fokker-Planck equation 
,  , Avw) 2 Ore 


When the representative point reaches the region where 7 > 0, 
the function F(v) comes into play, and reflects the representative 
point back into the region where v < 0. In fact, there remains 
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only a very small probability w,< 1! of penetration into the 
region wherc v > 0. Therefore, we can assume that the probability 
current (sce p. 62) through the boundary v = 0 vanishes, i-e., 


id =0 for v=0, 


Cv 


— yuw — yo" 
which leads to the boundary condition 


ow (0) _ 
=O: (9.125) 


This condition is not satisfied by the function (9.123); the solution 
of (9.124) which does satisfy (9.125) has the form 


1 1 
t St Pee ee 
wr, | Up) 4/ Ire a/} a e-2vt 


(9.126) 
1. (v — ve7”")?) I ot ee mye 
x [exp sa er eer ay ' l + exp | A ka Ge oi || 
(v<0,% <0). 
Then at the origin we have 
ot 2 1 we! ) 
wO, t | v9) Mag Eee ~ 9g? T — eat ; (9.127) 


and the function is extended in a continuous fashion into the 
region v > Q. In this region, the distribution is practically the same 
as the stationary distribution 


2 Lote 27! 
wv, t | Vp) = Vino VP nent rt exp | _ 72 T—e2v = mil 
(9.128) 
2 Rp 
x exp - — [ Fla) da| 
(z > 0 » Vg < 0) ’ 


which has the same value at the origin as (9.126). 
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If the initial point approaches the houndiary, i¢.. 7 »>O, the 
function (9.126), (9.128) goes into 


: a : ; wee 
wv, t | 0) wav} —e" Ee oe eer fore 2, 
v, 
; i eee eee ee 
eae} Io? nt {Exe vals tos wv tt) 
(9.129) 


When the initial valuc v, lies in the region where 7 is positive 
(so that z) > 0), then, due to the influence of (7), the represen- 
tative point is rapidly “pushed out” into the region where @ is 
negative. After this occurs, the distribution {| zy) 1s close to the 
distribution (9.129) just found. In fact, the two distributions differ 
only during a time of order (yRF’)"' =. R;C, which is negligibly 
small compared to 1/y = RC, because of the assumption nade 
above. 

Once we know the one-dimensional distribution (9.111) and the 
transition probability w(v, t | v)), we can write down the two- 
dimensional probability density 


w(v, U,) = wv) (v,, 7 | t) 


and the second moment 
(F(v) F(v,)> = f [ Fo) F(o,) w(v) w(v,, 717) dede,. (9.130) 
0 0 
Taking w(2,, 7 { v) in (9.130) to be (9.129), we find that 
! ue 1 
Mai) Sa Al = Se 
FO) Fed =H Vaan 
“F(e) exp | — 2 — 2 [r(ey det ae] 9.131 
«Lf (v) iz ael, (3) dz, at (9.131) 


et 
=n? Fe e eyt (rn O). 
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Because of (9.119) and (9.120), this gives 


(ays TON Po) ak 
kyr) = <F(o) (eg) — Mey? ~ = | aa 1]. (9.132) 
lor small + < R;C, the expression (9.132) is not valid, and for 
7 = 0, we can use the one-dimensional distribution (9.t 11), which 
leads to 


k,(0) = DF(v) = ae! | : exp - 2 [ F(z) dz! F2(y) dv. (9.133) 

‘l'o find the spectral density S[F(v) — <F); w] for frequencies w 
of order y and smaller, we can take the Fouricr transform of (9.132). 
Vhe region of small values of 7, where (9.132) is not valid, will 
have an important effect only for high frequencies w ~ 1/R;C. The 
lourier transform of (9.131) can be found by using the obvious 
relation 


is enn Brees - i dr =2Re i er lao — i] dr, 


where p = 7w, and the formula? 


. dr bpe dt 1 pip t 
2) a eee /2 ee — _ 
fe : V1 = entre 2y i e : V1 7 et dy 2 (5 : 5] : 
(9.134) 


where B(x, v) is the beta function. Comparing the last two 
equations, we sce that 


in tor I — I tw I — 2y 
i Ez -- e-2vitl i] ue 2 Re[B (5 5) tw)? 
and hence, according to (2.11) and (9.132), 
sis es eS iwRC | 
SIR) Fy se) = > Re B ( 5) (9135) 


See 1. M. Ryshik and 1. S. Gradstein, op. crt., formula (5.218.3), p. 258. 
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‘To determine the spectral density numerically, we can use 
tables of the gamma function, recalling that 


Pts 
. Biiz,4 =k Vn aS 


or else we can use the expansion 


. (2k -- 1)! | 
B (22, 3) = ras “Deh TRI(k - ave ADE wilRk 


To tmprove the convergence of the last formula, we transform it 
into4 
(2k — 1)! 
—s Fis Be eho 
B (iz, $) San 2In2 — iz > ae PRigik ijl Mie FB’ 


which implies that 


_1)f 
Re B(iz,}) =2In2 —2 2 oa ae WT Ke pay: 0-38 


Substituting (9.136) into (9.135), we have 


ae fa In2 — 2? [5 ster 


4 

S[FQ) —-<F> 50] =" 
ee 

+a rie ao a5 jj. 


4 Use thes fact that 
(2k — 1)! 


KJ 
2 In 2 = 2y pet 


which follows from 


1 = 1/2 
2In2 ~~ f — ey 
0 


x 


if the integrand is expanded in a power series and integrated term by term. 
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where 2 == w/2y. This function, which gives the spectral density of 
the fluctuations of the current J = F(v), is plotted in Figure 15. 


y= ote S [F(v)-(F);a| 


Fig. 15. The spectral density of the fluctuations of the current [| = F(v). 


According to (9.101), the output voltage 7 is obtained from F(z) 
by a linear transformation. It follows from the usual rules and 
(9.135) that the spectral density of the fluctuations of the output 
voltage is 


~2 
Sly — <n) sa] = [iw +E] STF) — Fs] 


(9.137) 
4, KC 


a” PREC? tT 


iwRC 1 
Re B (- ’ 5) . 
Finally, according to (2.15), to find the variance Dy, we have to 
integrate (9.137) with respect to the frequency®: 


Di = Sof. Re B (3) go = (1 —?) o? ~ 0.363 0%. (9.138) 


5 'To evaluate the integral in (9.138), we use (9.136) and the easily verified 
formula 


[=a md 
o (2 + 1) (x? + 4k) 22k FI’ 
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5.2. When the noise correlation time |-y ts differcnt from RC, we 
have to solve the two-dimensionel Fokker-Planck eqpiiten (9.105). 
This can be done by using methods similar to those given in 
Chap. 4, Sec. tf. Thus, to find the stattonary distribution satis- 
fying the equation 


2. 


Zee a fa 
es Ellie | bMo]e) te TS 


(9.139) 
a Cw - cre 
+ aE (Ew) | e€ ren | 2yo reek 0, 
where 
] 
oY RC 
we look for a solution of the form 
uv, E) — >, Tinnt "a@) xX AE). (9.140) 
min—- 
Here, the functions 
| E 
X,(€) = —— Fo» (5 9.141 
(¢) a Vn! (") ( ) 
are the eigenfunctions of equation (4.68), which satisfy the relations 
_ Van vn+1 
PO ag eR (9.142) 
EX, = —o[Vn X,., + Vn 1 1X). 


obtaining 
| dx = (2k 1)! ] 
- nil es ‘ 
ie ne ea(S, ag as 7 [In 2 2 zi RUR Vy! 22k 1 1) 


According to I. M. Ryshik and [. S. Gradstein, op. cit., formulas (1.641.1) 
and (1.642.2), p. 48, 


> (2e— Wt xt 
Lt 22k R — 1)E 2R(2k + 1) 


2 J . 
xin (= — arc cosh --) aresiny | x 
x x 


forx < I. If we set x — 1, (9.138) now follows from the last two formulas. 
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The eigenfunctions V,,,(v) satisfy the equation 
ey Cup wu l 
oor Mh = ae - f(g a — 
wile 2.1 keto] ra) bare <0. 08 
Substituting (9.140) into (9.139), and taking into account (9.142) 
and (9.143), we obtain 
= py Aig Canin derek ge a 4 > nT in VinXy 


rn myn 


— €0 > TannV nln Xy-1 -f- Vn 4-1 x ri 


yor, 7 


[- 2yo »; Dakin Vn -F 1 44 = 0, 


nnn 


(9.144) 


Next, we expand the derivatives V, with respect to the cigen- 
functions V,,, by writing 


— 7 
r. 7. > b mak ’ 
m 


where 
LA a (9.145) 
Then, from (9.144) we obtain the equations 


(Ayn | yn) Lik a (2 — €) o Vn DY ahi le ce 
hk 
(9.146) 
Sete S hash: 
k 


which can be solved by various methods. Having specified a given 
accuracy, we need only consider a certain number of expansion 
coefficients 7°,,,, of low order, setting the higher-order coefficients 
equal to zero. ‘Then, (9.146) gives a finite system of linear equations, 
from which we can determine the 7’,,,,. Another way of solving the 
problem is to use the method of successive approximations, 
regarding « as a small parameter, and obtaining the solution 
in the form of an expansion in powers of this parameter. 
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x Comhining the expressions obtained from (9.146) for » = 0, 
= | anda = 2, we find that 


i er oq Ur e(2y — €) a? »3 bias Tro 
aa: 
(9.147) 
r, 
+ €(2y — ej? of > One > = on boty, 4 Oe) (m +O), 
hyO,t 


where we have written 


Pie ] By ik 
i wn ; d; oe y 

By repeatedly substituting the whole right-hand side of (9.147) for 
Ti0, T 19, -- We can express the cocfficients T,,9 in terms of Toy: 


Divo = e(2y | €) Ody Lp 
(9.148) 
By + Dy: 


sat aa Dok Saris by Tyg 2 Ole) (mn = 0). 


To determine the other coefficients T,,,, we again turp to the 
equations (9,146), applying the method of successive approxima- 
tions. Letting 7°) denote the zeroth approximation, i.c., 

— T° | Ole), 


yk 
we find from (9.146) that 


Try = (2y — oy a Too, 


2 
CO) Jus ee ee LS "g) 
thee — (2y €) % Xm + 2y > Ba 4} (9.149) 


4 


— — cl q2 Burro 
= (2y €) oO ay Dit 4 y Loos 
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The zeroth approximation is then used to find the first approxima- 
tion to T‘?., which differs from T,,, by terms of order e*: 


na 
aiighx. oh (2y —- «)o V2 
Ta = 7 + P92 Wo Ty — cP Da MD, 


a Am + Y K#0 mt Y 


WW) a). 10) 
T= Qy—6)¢ wa 2 a, eu a_, 


mh” kl Am + 2y 7 mk” k3 ? 


(9. £50) 


This method allows us to find T,,, to within terms of order ¢*, 
where yp is any integer. Naturally, the larger x, the more formidable 
the calculations become. 

All the coefficients T,,,,, in the expansion (9.140) turn out to be 
proportional to the first coefficient T 9), which is determined by the 
normalization coefficient. In fact, if we integrate (9.140) with 
respect to v and &, all the terms except the first vanish, because of 
(4.54), and hence 


| w(v, £) dvdé = Ty = 1. (9.151) 
To simplify the catculations, it is convenient to assume (as on 
pp.274-277) that the function F(v) vanishes for v <0 and grows 
rapidly for v > 0. More precisely, tet R, = (OF /dv) -}, the internal 
resistance of the diode in its conducting state, satisfy the conditions 
R, <R, RC<. (9.152) 
Then, in considering the region v > 0, we can neglect the terms 
v ] 
re Re) § 


in the first of the equations (9.104), in comparison with F(v)/C; 
instead, we can use the equation 


b= — AF) +6. (9.153) 


SEC. 5} NONLINEAR TRANSFORMATIONS WITH MEMORY 285 


The corresponding Fokker-Planck equation has the stationary 
solution 

1 
ye . 


w(v) — const - exp | [" F(z) ds ; (9.154) 
0 
which is “established” after a time exceeding the sinall time 
constant R,C. 
For the region v <0, we can replace the function /(7) in 
equations (9.104) and (9.105), by the function 


) _ §0 for v<O, 
sis aa for vo >O. 


(9.155) 


The infinite value of F,(v) for v > 0 means that the represcntative 
point is immediately reflected back when it reaches the boundary 
v = 0. Actually, the reflection takes a time of order R,C, which ts 
considerably less than the time constants 1/RC and 1 y which 
characterize the process for v < 0. 

After replacing F(v) by (9.155), equation (9.143) has the eigen- 
functions 


1 


at re ee v 
Vim)! o VyRC 


as oVyRE 


Femsp ( ) (7 < QO) (9.156) 


[see formula (4.70)], and the eigenvalues 


2m ° 
An = 5a: 
According to (9.145), in the present case, using the foriula® 


ik Aan (2) Ao yi(2) € OP dz 
(2m — 1)! (2k -- It 


= (— hewn aes iia sae 


DK (RI (2m 2k bY)’ 


° Cf. Bateman Manuscript Project, op. crt., formula (9), p. 117; formula (21), 
p. 122; formula (7), p. 3. 
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we have 


tg ee = et 


ao V2ayRC V (2m)! (2k)! 2" me YE RE (2m -- 2k — 1) 
(9.157) 


oJ 


If we use (9.146) to calculate 7',,,, to some desired accuracy, then, 
according to (9.140), we obtain the jaint stationary probability 
density of the input and output signals, Le., 


Wels §) = [WY Eye = > TiinVYial& — 7) X,(€), (9.158) 


mn 


where the functions V,, and X,, are given by formulas (9.156) and 
(9.41). By using expansions of this kind, we can also investigate 
nonstationary transient processes, and two-dimensional probability 
densities corresponding to different instants of time. 
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